A Bayesian Information Criterion for Singular Models

Drton, Mathias; Plummer, Martyn

doi:10.1111/rssb.12187

Cited by 106 publications

(111 citation statements)

References 146 publications

(274 reference statements)

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…Section offers a comparison with applying overfitted mixtures to these data sets. Section reproduces a binomial mixture example that was used by Drton and Plummer () to illustrate sBIC, and Section analyses a US political blog data set via product binomial mixtures. We used R package NLPmix for the EM algorithm and the estimate

\overset{false^}{p} false(boldy false| M_{k} false)

from Marin and Robert (), and for our ECP estimator we used bfnormmix from R package mombf.…”

Section: Resultsmentioning

confidence: 99%

“…From a frequentist perspective the likelihood ratio test between

M_{k}

and

M_{k + 1}

may diverge as n →∞ when data truly arise from

M_{k}

unless restrictions on the parameters or likelihood penalties are imposed (Ghosh and Sen, ; Liu and Shao, ; Chen and Li, ). As an alternative one may consider criteria such as the Bayesian information criterion (BIC), Akaike's information criterion (AIC), the integrated complete likelihood (Biernacki et al ., ) or the singular BIC (Drton and Plummer, ), sBIC. Although the BIC justification as an approximation to the Bayesian evidence (Schwarz, ) is not valid for overfitted mixtures, it is often adopted as a useful criterion (Fraley and Raftery, ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On Choosing Mixture Components via Non-Local Priors

Fuquene

Steel

Rossell

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary Choosing the number of mixture components remains an elusive challenge. Model selection criteria can be either overly liberal or conservative and return poorly separated components of limited practical use. We formalize non‐local priors (NLPs) for mixtures and show how they lead to well‐separated components with non‐negligible weight, interpretable as distinct subpopulations. We also propose an estimator for posterior model probabilities under local priors and NLPs, showing that Bayes factors are ratios of posterior‐to‐prior empty cluster probabilities. The estimator is widely applicable and helps to set thresholds to drop unoccupied components in overfitted mixtures. We suggest default prior parameters based on multimodality for normal–T‐mixtures and minimal informativeness for categorical outcomes. We characterize theoretically the NLP‐induced sparsity, derive tractable expressions and algorithms. We fully develop normal, binomial and product binomial mixtures but the theory, computation and principles hold more generally. We observed a serious lack of sensitivity of the Bayesian information criterion, insufficient parsimony of the Akaike information criterion and a local prior, and a mixed behaviour of the singular Bayesian information criterion. We also considered overfitted mixtures; their performance was competitive but depended on tuning parameters. Under our default prior elicitation NLPs offered a good compromise between sparsity and power to detect meaningfully separated components.

show abstract

\overset{false^}{p} false(boldy false| M_{k} false)

from Marin and Robert (), and for our ECP estimator we used bfnormmix from R package mombf.…”

Section: Resultsmentioning

confidence: 99%

“…From a frequentist perspective the likelihood ratio test between

M_{k}

and

M_{k + 1}

may diverge as n →∞ when data truly arise from

M_{k}

Section: Introductionmentioning

confidence: 99%

On Choosing Mixture Components via Non-Local Priors

Fuquene

Steel

Rossell

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

show abstract

“…In the chart where a 2 = s, a 1 = sa 1 , b i = sb i , c i = sc i , d i = sd i , we do not have obstructions coming from any b * i = c * j = 0, v * i = 0, u * j = 0 so it is again easy to find the vector ξ for Lemma 3. The threshold is exactly (8,1).…”

Section: Appendicesmentioning

confidence: 99%

“…Thus, we shift our focus to the other charts of the blowup where the pullback pair is (s; s 7 ), so the RLCT is at least (8,1). In the chart where a 2 = s, a 1 = sa 1 , b i = sb i , c i = sc i , d i = sd i , we do not have obstructions coming from any b * i = c * j = 0, v * i = 0, u * j = 0 so it is again easy to find the vector ξ for Lemma 3.…”

Section: Appendicesmentioning

confidence: 99%

“…Nonetheless, our next theorem proves that the learning coefficient is always smaller than the (9/2, 1) prescribed by the BIC. For deeper statistical discussions, the reader should turn to Drton and Plummer [8] where they addressed the paradox of circular reasoning in requiring true parameter values for the asymptotic approximation of Bayesian integrals. They also proposed a novel algorithm where the marginal likelihood is estimated as a weighted average of the contributions from all true distributions.…”

Section: Applications To Statistical Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Ideal-Theoretic Strategies for Asymptotic Approximation of Marginal Likelihood Integrals

Lin¹

2017

J Alg Stat

View full text Add to dashboard Cite

Abstract. The accurate asymptotic evaluation of marginal likelihood integrals is a fundamental problem in Bayesian statistics. Following the approach introduced by Watanabe, we translate this into a problem of computational algebraic geometry, namely, to determine the real log canonical threshold of a polynomial ideal, and we present effective methods for solving this problem. Our results are based on resolution of singularities. They apply to parametric models where the Kullback-Leibler distance is upper and lower bounded by scalar multiples of some sum of squared real analytic functions. Such models include finite state discrete models.

show abstract