Optimal compromise between incompatible conditional probability distributions, with application to Objective Bayesian Kriging

Abstract

Models are often defined through conditional rather than joint distributions, but it can be difficult to check whether the conditional distributions are compatible, i.e. whether there exists a joint probability distribution which generates them. When they are compatible, a Gibbs sampler can be used to sample from this joint distribution. When they are not, the Gibbs sampling algorithm may still be applied, resulting in a "pseudo-Gibbs sampler". We show its stationary probability distribution to be the optimal compromise between the conditional distributions, in the sense that it minimizes a mean squared misfit between them and its own conditional distributions. This allows us to perform Objective Bayesian analysis of correlation parameters in Kriging models by using univariate conditional Jeffreys-rule posterior distributions instead of the widely used multivariate Jeffreys-rule posterior. This strategy makes the full-Bayesian procedure tractable. Numerical examples show it has near-optimal frequentist performance in terms of prediction interval coverage.Résumé. Les modèles statistiques sont souvent définis par lois de probabilité conditionnelles plutôt que jointes. Il peut cependant être difficile de contrôler la compatibilité des lois conditionnelles, i.e. l'existence d'une loi jointe qui les engendre. Quand les lois conditionnelles sont compatibles, un échantillonneur de Gibbs peut être utilisé pour échantillonner selon la loi jointe dont elles procèdent. Dans le cas contraire, l'algorithme de l'échantillonneur de Gibbs peut parfois tout de même être appliqué. Une telle procédure est appelée « pseudo-échantillonneur de Gibbs ». Nous montrons que sa distribution stationnaire est le compromis optimal entre les lois conditionnelles, au sens où elle minimise un écart quadratique moyen entre elles et ses propres lois conditionnelles. Cela permet une analyse bayésienne objective des paramètres de corrélation de modèles de krigeage : nous utilisons des lois a posteriori de Jeffreys univariées conditionnelles plutôt que la très usitée loi a posteriori de Jeffreys multivariée. Cette stratégie rend la procédure pleinement bayésienne abordable. Des exemples numériques montrent que ses performances fréquentistes en termes de taux de couverture des intervalles prédictifs sont quasi-optimales.

Probabilistic surrogate modeling by Gaussian process: A review on recent insights in estimation and validation

“…Note that an alternative solution (not detailed here) is proposed by Muré, J. [79] to make the full-Bayesian procedure tractable: univariate conditional Jeffreys-rule posterior distributions and pseudo-Gibbs sampler are notably used.…”

Named Entity Recognition in Equipment Support Field Using Tri-Training Algorithm and Text Information Extraction Technology

“…Obviously, the advantages of Tri-training semi-supervised learning are that it can classify with a small amount of labeled corpus and a large number of the unlabelled corpus, the military equipment support information is mostly classified as confidential, and labeled corpus is difficult to obtain. Therefore, the semisupervised learning Tri-training algorithm is applied to train a better classification model on fewer labeled corpus and a large number of unlabelled corpus [29][30][31].…”

Integrated organic inference (IOI): A reconciliation of statistical paradigms

“…Similar to what was done earlier, here we will again focus attention on a more widely applicable method, in particular the method that simply consists in making the assumption that the joint density of the parameters θ that most closely corresponds to the set of full conditional densities in equation ( 2) is equal to the limiting density function of a Gibbs sampling algorithm that is based on these conditional densities with some given fixed or random scanning order of the parameters concerned. This approach relates to more specific methods for addressing the general problem of interest that were discussed in Chen, Ip and Wang (2011) and Muré (2019).…”