Online Learning with Continuous Ranked Probability Score

Abstract

We consider the case where several competing methods produce online predictions in the form of probability distribution functions. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is Continuous Ranked Probability Score (CRPS). In this paper the problem of combining probabilistic forecasts is considered in the prediction with expert advice framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk's aggregating algorithm using CRPS as a loss function can be obtained. Also, we incorporate in this scheme a smooth version of the method of specialized experts which allows us to make more flexible and accurate predictions. We present the results of numerical experiments illustrating the proposed methods.