Objective Bayesian inference with proper scoring rules

Abstract

Standard Bayesian analyses can be difficult to perform when the full likelihood, and consequently the full posterior distribution, is too complex or even impossible to specify or if robustness with respect to data or to model misspecifications is required. In these situations, we suggest to resort to a posterior distribution for the parameter of interest based on proper scoring rules. Scoring rules are loss functions designed to measure the quality of a probability distribution for a random variable, given its observed value. Important examples are the Tsallis score and the Hyvärinen score, which allow us to deal with model misspecifications or with complex models. Also the full and the composite likelihoods are both special instances of scoring rules.The aim of this paper is twofold. Firstly, we discuss the use of scoring rules in the Bayes formula in order to compute a posterior distribution, named SR-posterior distribution, and we derive its asymptotic normality. Secondly, we propose a procedure for building default priors for the unknown parameter of interest, that can be used to update the information provided by the scoring rule in the SR-posterior distribution. In particular, a reference prior is obtained by maximizing the average α−divergence from the SR-posterior distribution. For 0 ≤ |α| < 1, the result is a Jeffreys-type prior that is proportional to the square root of the determinant of the Godambe information matrix associated to the scoring rule. Some examples are discussed.

Robust inference for non‐linear regression models from the Tsallis score: Application to coronavirus disease 2019 contagion in Italy

“…The proposed SR‐posterior distribution is defined as with , where C is a d × d fixed matrix such that C T K ( θ ) C = G ( θ ). A possible choice of the matrix C is given by C = M −1 M A , with M T M A = G and M T M = K ; for details, see Giummolé et al (2019) and references therein.…”

“…The choice of a prior distribution π ( θ ) to be used in (9) involves the same problems typical of the standard Bayesian perspective. For instance, for objective Bayesian inference, the expected α ‐divergence to the Tsallis posterior distribution can be used (Giummolé et al, 2019), and it is given by π G ( θ ) ∝ | G ( θ )| 1/2 .…”

“…Figure 6 gives the violin plots of the marginal posterior distributions for parameters of the model and the expected mean, both for DD and for ICU data, using the reference prior (Giummolé et al, 2019). The posterior medians (the white dots) for the upper asymptotes and the inflection points are consistent with the frequentist robust estimates.…”

“…This paper aims to discuss the use of robust inference on non‐linear regression models. In particular, we discuss a general approach based on the Tsallis score (Basu, Harris, Hjort, & Jones, 1998; Dawid, Musio, & Ventura, 2016; Ghosh & Basu, 2013; Giummolé, Mameli, Ruli, & Ventura, 2019; Mameli, Musio, & Ventura, 2018), in both frequentist and Bayesian frameworks.…”

On Default Priors for Robust Bayesian Estimation with Divergences

“…For example, if we assume the -contamination model (see, e.g., [ 1 ]) as a data generating distribution, many objective priors depend on the unknown contamination ratio and unknown contamination distribution because these objective priors involve the expectations under the data generating distribution. Although [ 17 ] derived some kinds of reference priors under the quasi-posterior distributions based on some kinds of scoring rules, they only discussed the robustness of such reference priors when the contamination ratio is approximately zero. Furthermore, their simulation studies largely depended on the assumption for the contamination ratio.…”