Prior Sample Size Extensions for Assessing Prior Impact and Prior-Likelihood Discordance

Abstract

This paper outlines a framework for quantifying the prior's contribution to posterior inference in the presence of priorlikelihood discordance, a broader concept than the usual notion of prior-likelihood conflict. We achieve this dual purpose by extending the classic notion of prior sample size, M, in three directions: (I) estimating M beyond conjugate families; (II) formulating M as a relative notion that is as a function of the likelihood sample size k, M(k), which also leads naturally to a graphical diagnosis; and (III) permitting negative M, as a measure of prior-likelihood conflict, that is, harmful discordance. Our asymptotic regime permits the prior sample size to grow with the likelihood data size, hence making asymptotic arguments meaningful for investigating the impact of the prior relative to that of likelihood. It leads to a simple asymptotic formula for quantifying the impact of a proper prior that only involves computing a centrality and a spread measure of the prior and the posterior. We use simulated and real data to illustrate the potential of the proposed framework, including quantifying how weak is a 'weakly informative' prior adopted in a study of lupus nephritis. Whereas we take a pragmatic perspective in assessing the impact of a prior on a given inference problem under a specific evaluative metric, we also touch upon conceptual and theoretical issues such as using improper priors and permitting priors with asymptotically non-vanishing influence.

Quantification of empirical determinacy: the impact of likelihood weighting on posterior location and spread in Bayesian meta-analysis estimated with JAGS and INLA

“…These posterior descriptive statistics are provided by default by generalpurpose software for Bayesian computation. Moreover, they are used by other modern approaches to quantify the impact of priors on posteriors [Reimherr et al, 2021]. However, for stability reasons, recommend the use of location and spread estimates based on quantiles.…”

Calibrated Model Criticism Using Split Predictive Checks

“…Effect of the prior Finally, we investigate the effect of the prior on the test size of the predictive checks. Conjugate exponential families lend themselves to the definition of a prior effective sample size (ESS) (though see Reimherr, Meng and Nicolae (2021) for an extension to other models). For a Poisson model with a Gamma(α, β) prior, the prior ESS is N 0 = β.…”

Six Statistical Senses

“…This is because there is no probability distribution to describe ignorance: any probability distribution specification about θ encodes restrictions on how likely one possible state of θ is versus another, and hence it cannot represent ignorance. Many attempts have been made in the literature to find the so-called "non-informative distributions" (see Kass & Wasserman, 1996, for an overview), but they inevitably lead to theoretical and/or logical inconsistencies even if they may provide acceptable practical answers (see Reimherr et al, 2021, for an example). The situation is a bit like doing arithmetic without the number zero, and uses some other (small) numbers to represent zero, which may be acceptable in some practical cases but logical inconsistencies and paradoxes would inevitably ensue (e.g., only a true zero remains zero after multiplication).…”