Bayesian Model Specification: Heuristics And Examples

draper, david

doi:10.1093/acprof:oso/9780199695607.003.0020

Cited by 20 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Leamer [1978] is a notable early effort advocating ad-hoc model selection for the purpose of human comprehensibility. Fouskakis and Draper [2008], Fouskakis et al [2009] and Draper [2013] represent efforts to define variable importance in real-world terms using subject matter considerations. A more generic approach is to gauge predictive relevance [Gelfand et al, 1992].…”

mentioning

confidence: 99%

Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective

Hahn¹,

Carvalho

2015

Journal of the American Statistical Association

150

191

View full text Add to dashboard Cite

Selecting a subset of variables for linear models remains an active area of research. This paper reviews many of the recent contributions to the Bayesian model selection and shrinkage prior literature. A posterior variable selection summary is proposed, which distills a full posterior distribution over regression coefficients into a sequence of sparse linear predictors.1. Introduction. This paper revisits the venerable problem of variable selection in linear models. The vantage point throughout is Bayesian: a normal likelihood is assumed and inferences are based on the posterior distribution, which is arrived at by conditioning on observed data.In applied regression analysis, a "high-dimensional" linear model can be one which involves tens or hundreds of variables, especially when seeking to compute a full Bayesian posterior distribution.Our review will be from the perspective of a data analyst facing a problem in this "moderate" regime. Likewise, we focus on the situation where the number of predictor variables, p, is fixed.In contrast to other recent papers surveying the large body of literature on Bayesian variable selection [Liang et al., 2008, Bayarri et al., 2012 and shrinkage priors [O'Hara and Sillanpää, 2009, Polson andScott, 2012], our review focuses specifically on the relationship between variable selection priors and shrinkage priors. Selection priors and shrinkage priors are related both by the statistical ends they attempt to serve (e.g., strong regularization and efficient estimation) and also in the technical means they use to achieve these goals (hierarchical priors with local scale parameters).We also compare these approaches on computational considerations.Finally, we turn to variable selection as a problem of posterior summarization. We argue that if variable selection is desired primarily for parsimonious communication of linear trends in the data, that this can be accomplished as a post-inference operation irrespective of the choice of prior distribution. To this end, we introduce a posterior variable selection summary, which distills a full

show abstract

mentioning

confidence: 99%

Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective

Hahn¹,

Carvalho

2015

Journal of the American Statistical Association

150

191

View full text Add to dashboard Cite

show abstract

“…Bernardo and Smith ()), posterior model probabilities correspond to a utility function—often somewhat strange in practice—in which the decision maker receives 1 utile for choosing the correct model in a finite set

{{scriptM}_{i}, i \in I}

of candidates and 0 utiles otherwise. Other, arguably more natural, utility functions lead to other model comparison strategies; a leading example is log‐scoring , arising from a utility function that rewards predictive accuracy (see, for example, Gelfand and Ghosh () and Draper () for details). As the authors emphasize, the asymptotics in this paper are based on a scenario in which the set

{{scriptM}_{i}, i \in I}

of candidate models is held constant and the sample size n is permitted to increase without bound. But surely this is unrealistic for actual practice: as n grows, a good statistician will typically enlarge

{{scriptM}_{i}, i \in I}

by enriching it with more complicated models, to do a better job of emulating the complexity of the real world.…”

Section: Discussion On the Paper By Drton And Plummermentioning

confidence: 99%

A Bayesian Information Criterion for Singular Models

Drton

Plummer

2017

Journal of the Royal Statistical Society Series B: Statistical Methodology

106

111

View full text Add to dashboard Cite

Abstract. We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher-information matrices may fail to be invertible along other competing submodels. Such singular models do not obey the regularity conditions underlying the derivation of Schwarz's Bayesian information criterion (BIC) and the penalty structure in BIC generally does not reflect the frequentist large-sample behavior of their marginal likelihood. While large-sample theory for the marginal likelihood of singular models has been developed recently, the resulting approximations depend on the true parameter value and lead to a paradox of circular reasoning. Guided by examples such as determining the number of components of mixture models, the number of factors in latent factor models or the rank in reduced-rank regression, we propose a resolution to this paradox and give a practical extension of BIC for singular model selection problems.

show abstract

“…Thus, a good model would be one that provides a large value of p(z new |z, ⌰). Proper rules for comparing a data value z new with its predictive distribution involve the logarithm of the height of p(z new |z, ⌰), or log(p(z new |z, ⌰)) (Gneiting and Raftery 2007;Krnjajić et al 2008;Draper and Krnjajić 2010;Draper 2013). This metric of predictive quality is known as the log-score (LS).…”

Section: Model Selection For the Habitat Modelmentioning

confidence: 99%

“…This suggests we would need to run a number of MCMC models for each covariate and each run would have a different set of data points excluded from model estimation (e.g., Draper and Krnjajić 2010;Shelton et al 2012;Draper 2013). In practice, this is impractical due to the long computing times for models estimated with MCMC.…”

Section: Model Selection For the Habitat Modelmentioning

confidence: 99%

“…In practice, this is impractical due to the long computing times for models estimated with MCMC. Fortunately, with reasonably large sample sizes, we can use what is known as the full sample log-score that will approximate the cross-validation derived log-score (Krnjajić et al 2008;Draper and Krnjajić 2010;Draper 2013). For each draw of the MCMC, g, we calculate the predicted probability of each observed data point, i, then…”

Section: Model Selection For the Habitat Modelmentioning

confidence: 99%

See 1 more Smart Citation

Spatial semiparametric models improve estimates of species abundance and distribution

Olaf

ThorsonJames

WardEric

et al. 2014

Can. J. Fish. Aquat. Sci.

134

112

View full text Add to dashboard Cite

Accurate estimates of abundance are imperative for successful conservation and management. Classical, stratified abundance estimators provide unbiased estimates of abundance, but such estimators may be imprecise and impede assessment of population status and trend when the distribution of individuals is highly variable in space. Model-based procedures that account for important environmental covariates can improve overall precision, but frequently there is uncertainty about the contribution of particular environmental variables and a lack of information about variables that are important determinants of abundance. We develop a general semiparametric mixture model that incorporates measured habitat variables and a nonparametric smoothing term to account for unmeasured variables. We contrast this spatial habitat approach with two stratified abundance estimators and compare the three models using an intensively managed marine fish, darkblotched rockfish (Sebastes crameri). We show that the spatial habitat model yields more precise, biologically reasonable, and interpretable estimates of abundance than the classical methods. Our results suggest that while design-based estimators are unbiased, they may exaggerate temporal variability of populations and strongly influence inference about population trend. Furthermore, when such estimates are used in broader meta-analyses, such imprecision may affect the broader biological inference (e.g., the causes and consequences of the variability of populations).Résumé : Des estimations exactes de l'abondance sont essentielles au succès de la conservation et de la gestion. Si les estimateurs d'abondance stratifiés classiques fournissent des estimations non biaisées de l'abondance, ces estimateurs peuvent être imprécis ou entraver l'évaluation de l'état et de la tendance de la population si la répartition des individus est très variable dans l'espace. Si des procédures basées sur des modèles qui tiennent compte d'importantes covariables environnementales peuvent améliorer la précision globale, il y a souvent une incertitude associée à la contribution de différentes variables environnementales et un manque d'information sur les variables qui sont d'importants déterminants de l'abondance. Nous avons développé un modèle de mélange semi-paramétrique général qui incorpore des variables mesurées de l'habitat et un terme de lissage non paramé-trique pour tenir compte des variables non mesurées. Nous comparons cette approche d'habitat spatial à deux estimateurs d'abondance stratifiés à la lumière d'observations sur un poisson marin faisant l'objet d'une gestion intensive, le sébaste tacheté (Sebastes crameri). Nous démontrons que le modèle d'habitat spatial produit des estimations de l'abondance plus précises, interprétables et raisonnables du point de vue biologique que les méthodes classiques. Nos résultats donnent à penser que, si les estimateurs basés sur la conception de l'échantillonnage sont non biaisés, ils peuvent exagérer la variabilité temporelle des populations et influencer fo...

show abstract

Bayesian Model Specification: Heuristics And Examples

Cited by 20 publications

References 10 publications

Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective

Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective

A Bayesian Information Criterion for Singular Models

Spatial semiparametric models improve estimates of species abundance and distribution

Contact Info

Product

Resources

About