Split Conformal Prediction for Dependent Data

Abstract

Split conformal prediction is a popular tool to obtain predictive intervals from general statistical algorithms, with few assumptions beyond data exchangeability. We show that coverage guarantees from split CP can be extended to dependent processes, such as the class of stationary β-mixing processes, by adding a small coverage penalty. In particular, we show that the empirical coverage bounds for some β-mixing processes match the order of the bounds under exchangeability. The framework introduced also extends to non-stationary processes and to other CP methods, and experiments corroborate our split CP's coverage guarantees under dependent data.

Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data

“…Generally speaking, examples are assumed to be exchangeable in a CP context. Most pertinent to our work, (Gendler et al 2022;Tibshirani et al 2019;Fisch et al 2021;Cauchois et al 2020;Gibbs and Candès 2021;Oliveira et al 2022) all consider various situations in which the exchangeability of the examples is violated to some extent. (Gendler et al 2022) considers the case in which the test examples may be adversarially attacked (Szegedy et al 2014;Goodfellow, Shlens, and Szegedy 2015;Madry et al 2018); (Tibshirani et al 2019) investigates the situation in which the density ratio between the target domain and the source domain is known; (Fisch et al 2021) studies the few-shot learning setting and assumes that the source domains and the target domain are independent and identically distributed (i.i.d.)…”

“…(Gendler et al 2022) considers the case in which the test examples may be adversarially attacked (Szegedy et al 2014;Goodfellow, Shlens, and Szegedy 2015;Madry et al 2018); (Tibshirani et al 2019) investigates the situation in which the density ratio between the target domain and the source domain is known; (Fisch et al 2021) studies the few-shot learning setting and assumes that the source domains and the target domain are independent and identically distributed (i.i.d.) from some distribution on the domains; (Gibbs and Candès 2021) considers an online learning setting and (Oliveira et al 2022) provides results when the examples are mixing (Achim 2013;Xiaohong, Lars Peter, and Marine 2010;Bin 1994). Different from all the works discussed above, we consider the OOD generalization setting in which the f -divergence between the target domain and the convex hull of the source domains is constrained.…”