Bayesian model selection for spatial capture–recapture models

Abstract

A vast amount of ecological knowledge generated over the past two decades has hinged upon the ability of model selection methods to discriminate among various ecological hypotheses. The last decade has seen the rise of Bayesian hierarchical models in ecology. Consequently, commonly used tools, such as the AIC, become largely inapplicable and there appears to be no consensus about a particular model selection tool that can be universally applied. We focus on a specific class of competing Bayesian spatial capture–recapture (SCR) models and apply and evaluate some of the recommended Bayesian model selection tools: (1) Bayes Factor—using (a) Gelfand‐Dey and (b) harmonic mean methods, (2) Deviance Information Criterion (DIC), (3) Watanabe‐Akaike's Information Criterion (WAIC) and (4) posterior predictive loss criterion. In all, we evaluate 25 variants of model selection tools in our study. We evaluate these model selection tools from the standpoint of selecting the “true” model and parameter estimation. In all, we generate 120 simulated data sets using the true model and assess the frequency with which the true model is selected and how well the tool estimates N (population size), a parameter of much importance to ecologists. We find that when information content is low in the data, no particular model selection tool can be recommended to help realize, simultaneously, both the goals of model selection and parameter estimation. But, in general (when we consider both the objectives together), we recommend the use of our application of the Bayes Factor (Gelfand‐Dey with MAP approximation) for Bayesian SCR models. Our study highlights the point that although new model selection tools are emerging (e.g., WAIC) in the applied statistics literature, those tools based on sound theory even under approximation may still perform much better.

Resource pulses influence the spatio‐temporal dynamics of a large carnivore population

“…If necessary, we discarded more iterations as burn‐in until the Gelman–Rubin convergence diagnostic indicated adequate convergence ( ≤ 1.05). For each survey period, our model choice was influenced by a combination of the following three criteria (Elliot et al 2020): the Bayesian p‐value (Royle et al 2009), the logarithm of the marginal likelihood using the harmonic mean estimator (Dey et al 2019) and the pair‐wise correlation plots between estimated parameters from the posterior MCMC draws. We were particularly concerned about correlations with the density parameters.…”

The importance of reliable monitoring methods for the management of small, isolated populations

“…We were particularly concerned to explore if any correlation was influencing the abundance parameters; Finally, although there is no established model selection method proven to work well for Bayesian SECR models such as ours, a recent development using simulations shows promise (see equation 2.6, Dey, Delampady, & Gopalaswamy, 2019, noting that the SECR models used by them were different from ours). We considered the adequate models obtained from the first step and, as recommended in Dey et al (2019), we applied the harmonic mean estimator of the marginal likelihood using the MCMC draws to finally select the model generating the highest value for the logarithm of the marginal likelihood. As such, our model choice was influenced by all the criteria described above.…”

Relative Risk Estimation for Human Leptospirosis Disease in Malaysia Based on Existing Models and Discrete Space-Time Stochastic Sir Model

“…DIC is used to compare the models which best fit the data. However, some studies disputed the capacity of DIC to compare hierarchical or multi-level models when re-parameterisation occurs (Pooley & Marion, 2018;Dey et al, 2019). On the other hand, it is very elusive to find a study which compares DICs between different forms of mathematical models.…”