Conformal Prediction

最新推荐文章于 2026-06-17 23:49:13 发布

原创

最新推荐文章于 2026-06-17 23:49:13 发布 · 830 阅读

标签

#算法 #人工智能 #机器学习

博客围绕分类中的共形预测和大语言模型的共形事实保证展开。在分类共形预测方面，介绍了共形覆盖保证和自适应预测集分类的计算方法；对于大语言模型，阐述了如何构建具有共形事实保证的输出，通过定义共形分数和阈值来确保一定的正确性概率。

Conformal Prediction in Classification

Conformal Coverage Guarantee

Given the calibration data set $(Xi,Yi∗)}i=1n{\left \{\left ( X_i, {Y}_{i}^{*}\right ) \right \}}_{i=1}^{n}$ and pretrained model $f^(⋅)\hat{f}\left ( \cdot\right )$ ( $f^(Xi)∈[0,1](K)\hat{f}\left ( X_i\right ) \in {\left [ 0, 1\right ]}^{\left ( K\right )}$ ).
The probability (or confidence) assigned to the true label is $f^(Xi)Yi∗{\hat{f}\left ( X_i\right ) }_{ {Y}_{i}^{*}}$ .

Calculate and sort the conformal scores: $si=s(Xi,Yi∗)=1−f^(Xi)Yi∗s_i= s\left ( X_i, {Y}_{i}^{*}\right ) =1-{\hat{f}\left ( X_i\right ) }_{ {Y}_{i}^{*}}$ ( $s1≤⋯≤sn}\left \{s_1 \leq \cdots \leq s_n \right \}$ ).

Obtain the $⌈(n+1)(1−α)⌉n\frac{\left \lceil \left ( n+1\right )\left ( 1-\alpha \right )\right \rceil}{n}$ quantile of $si}i=1n{\left \{ s_i\right \}}_{i=1}^{n}$ : $q^=inf⁡{ q:∣{ i:si≤q}∣n≥⌈(n+1)(1−α)⌉n}=s⌈(n+1)(1−α)⌉\hat{q}=\inf \left \{ q:\frac{\left | \left \{ i:s_i \leq q\right \}\right |}{n} \geq \frac{\left \lceil \left ( n+1\right )\left ( 1-\alpha \right )\right \rceil}{n} \right \} = {s}_{\left \lceil \left ( n+1\right )\left ( 1-\alpha \right )\right \rceil}$ .

Construct the prediction set of $(Xtest,Ytest∗)\left ( {X}_{test}, {Y}_{test}^{*}\right )$ : $y:f^(Xtest)y≥1−q^}={ y:s(Xtest,y)≤q^}\mathcal{C}\left ( {X}_{test}\right )=\left \{ y: {\hat{f}\left ( {X}_{test}\right )}_{y} \geq 1-\hat{q} \right \}=\left \{ y: s\left ( {X}_{test}, y\right )\leq \hat{q}\right \}$ .

The event $Ytest∗∈C(Xtest)}\left \{ {Y}_{test}^{*} \in \mathcal{C}\left ( {X}_{test}\right ) \right \}$ is equivalent to $s(Xtest,Ytest∗)≤q^}\left \{ s\left ( {X}_{test}, {Y}_{test}^{*}\right )\leq \hat{q}\right \}$ .

By the exchangeability of $,(Xn,Yn),(Xtest,Ytest∗)\left ( X_1, Y_1\right ), \cdots ,\left ( X_n, Y_n\right ), \left ( {X}_{test}, {Y}_{test}^{*}\right )$

最低0.47元/天解锁文章