Improving Marginal Likelihood Estimation for Bayesian Phylogenetic Model Selection

Abstract

The marginal likelihood is commonly used for comparing different evolutionary models in Bayesian phylogenetics and is the central quantity used in computing Bayes Factors for comparing model fit. A popular method for estimating marginal likelihoods, the harmonic mean (HM) method, can be easily computed from the output of a Markov chain Monte Carlo analysis but often greatly overestimates the marginal likelihood. The thermodynamic integration (TI) method is much more accurate than the HM method but requires more computation. In this paper, we introduce a new method, steppingstone sampling (SS), which uses importance sampling to estimate each ratio in a series (the "stepping stones") bridging the posterior and prior distributions. We compare the performance of the SS approach to the TI and HM methods in simulation and using real data. We conclude that the greatly increased accuracy of the SS and TI methods argues for their use instead of the HM method, despite the extra computation needed.

Bayesian estimation of fossil phylogenies and the evolution of early to middle Paleozoic crinoids (Echinodermata)

“…Bayes factors are used in Bayesian model selection to determine which parameter configurations provide the best fit to the data and are equal to twice the difference in marginal loglikelihoods between models (Kass and Raftery, 1995). Following phylogenetic analyses, I then estimated the marginal log-likelihood of each model using the stepping-stone sampling method (Xie et al, 2010) with 50 steps and powers of β corresponding to quantiles of a Beta(0.5, 1.0) distribution. Parsimony-based calculations were performed using PAUP* 4.0a147 (Swofford, 2002).…”

Assessing species boundaries and the phylogenetic position of the rare Szechwan ratsnake, Euprepiophis perlaceus (Serpentes: Colubridae), using coalescent-based methods

“…Unlike the other model comparison approaches (likelihood ratio test [LRT], Akaike Information Criterion [AIC], and Bayesian Information Criterion [BIC], etc. ), BF calculates the ratio of the marginal likelihood of two models, which has the advantage of taking into account priors used in Bayesian analyses (Xie et al, 2011). The marginal likelihood values of these two competing models (Model 1, Model 2) were estimated using recently developed techniques including path sampling (PS, Lartillot and Philippe, 2006) and stepping-stone sampling (SS, Xie et al, 2011), which have previously demonstrated to have better performance than the harmonic mean estimator (HME, Newton and Calculations of PS and SS were performed in BEAST v 1.7.4.…”

“…), BF calculates the ratio of the marginal likelihood of two models, which has the advantage of taking into account priors used in Bayesian analyses (Xie et al, 2011). The marginal likelihood values of these two competing models (Model 1, Model 2) were estimated using recently developed techniques including path sampling (PS, Lartillot and Philippe, 2006) and stepping-stone sampling (SS, Xie et al, 2011), which have previously demonstrated to have better performance than the harmonic mean estimator (HME, Newton and Calculations of PS and SS were performed in BEAST v 1.7.4. First, we ran species tree analyses in * BEAST based on Model 1 and Model 2 respectively, using the same phased dataset for phylogenetic inference including all three lineages of Euprepiophis and five outgroup species.…”

“…Since incorrect phylogenetic relationships in the guide tree will mislead the results supporting lineage splitting with high Pp, a fully resolved and correct guide tree is necessary (Leaché and Fujita, 2010). Thus, the other two tests using Brownie and BF model comparison, without constraining the tree topology a priori were also used and yield similar results to BP&P. For the BF model comparison approach, among multiple marginal likelihood estimators used for calculating the likelihood values including harmonic mean estimator (HME), the stabilized/smoothed harmonic mean estimator (sHME, Redelings and Suchard, 2005), path sampling (PS), and stepping-stone sampling (SS), HME and sHME (an extension of HME) have been demonstrated to perform poorly in choosing the correct model (Baele et al, 2012a,b;Fan et al, 2011;Xie et al, 2011). Specifically, a simulation study comparing the power of HME, sHME, PS and SS for the species model selection in * BEAST shows that only PS and SS estimators are sensitive in identifying the correct species model with both over splitting or lumping of lineages (Grummer et al, in press).…”

Modeling and analysis of residential flexibility: Timing of white good usage

“…to the data, whereas likelihood based or point estimators such as Akaike information criteria (AIC) [33] or Bayesian information criteria (BIC) [34] base their decisions on the best fit of each competing model [35]. The use of the marginal likelihood also automatically penalizes the complexity of the model because complex models spread their probability mass widely by predicting various possible outcomes, hence the probability of actual data will be smaller for overly complex models.…”