Adaptive Conformal Predictions for Time Series

Abstract

Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs and Candès, 2021), developed for distribution-shift time series, is a good procedure for time series with general dependency. We theoretically analyse the impact of the learning rate on its efficiency in the exchangeable and auto-regressive case. We propose a parameter-free method, AgACI, that adaptively builds upon ACI based on online expert aggregation. We lead extensive fair simulations against competing methods that advocate for ACI's use in time series. We conduct a real case study: electricity price forecasting. The proposed aggregation algorithm provides efficient prediction intervals for day-ahead forecasting. All the code and data to reproduce the experiments is made available.

Conformalized Online Learning: Online Calibration Without a Holdout Set

“…Is online Has a coverage guarantee The model uses all data ACI [21] OSSCP [22] ACI-Online ( [21,22] and Supplementary Section E.3)…”

“…Faster reaction to distribution shift. In contrast to existing techniques [21,22], our proposal does not reserve a holdout set for calibrating the prediction interval at each inference step. This allows fitting the model on the entire data stream, and utilizing the most recent observations to better track the underlying distribution.…”

“…Also, splitting the data at random breaks the dependency structure across the time horizon, and this may reduce the predictive power when close-by data points are dependent. To address this issue and reduce the computational complexity, we combine ACI with online sequential split conformal prediction (OSSCP) [22], which can be conducted online, to form a strong, feasible baseline method.…”

“…The time-series data sets we use in the experiments in Section 5 include the day of the week as an element in the feature vector. Therefore, we assess the violation of day-stratified coverage [22,37], as a proxy for conditional coverage. That is, we measure the average deviation of the coverage in each day of the week from the nominal coverage level; see Supplementary Section F.2 for a formal definition.…”

“…The method of ACI-Online we introduce here is a straightforward combination of online sequential split conformal prediction (OSSCP) [22] and ACI, which we consider as a powerful baseline method.…”