Bayesian Criterion-Based Variable Selection

Abstract

Bayesian approaches for criterion based selection include the marginal likelihood based highest posterior model (HPM) and the deviance information criterion (DIC). The DIC is popular in practice as it can often be estimated from sampling‐based methods with relative ease and DIC is readily available in various Bayesian software. We find that sensitivity of DIC‐based selection can be high, in the range of 90–100%. However, correct selection by DIC can be in the range of 0–2%. These performances persist consistently with increase in sample size. We establish that both marginal likelihood and DIC asymptotically disfavour under‐fitted models, explaining the high sensitivities of both criteria. However, mis‐selection probability of DIC remains bounded below by a positive constant in linear models with g‐priors whereas mis‐selection probability by marginal likelihood converges to 0 under certain conditions. A consequence of our results is that not only the DIC cannot asymptotically differentiate between the data‐generating and an over‐fitted model, but, in fact, it cannot asymptotically differentiate between two over‐fitted models as well. We illustrate these results in multiple simulation studies and in a biomarker selection problem on cancer cachexia of non‐small cell lung cancer patients. We further study the performances of HPM and DIC in generalized linear model as practitioners often choose to use DIC that is readily available in software in such non‐conjugate settings.

Evaluating methods for measuring in‐river bathymetry: Remote sensing green LIDAR provides high‐resolution channel bed topography limited by water penetration capability

“…The Deviance Information Criterium was used for model selection (Ando, 2007; Maity et al, 2021; Ward, 2008). Model selection was based on scaled values, since scaling permits direct comparison of continuous variable predictor coefficients and their importance in the model (e.g., Schielzeth, 2010), and starting with a model including all relevant predictors and interactions.…”

Bayesian variable selection and survival modeling: Assessing the most important comorbidities that impact lung and colorectal cancer survival in Spain

“…4 Biomedical Network Research Centers of Epidemiology and Public Health (CIBERESP), Madrid, Spain. 5 Andalusian School of Public Health, Granada, Spain. 6 Epidemiology Unit and Girona Cancer Registry, Oncology Coordination Plan, Department of Health, Autonomous Government of Catalonia, Catalan Institute of Oncology, Girona, Spain.…”

“…Two limitations of this approach are their poor performance in terms of correct model selection for finite samples, and potential multiplicity problems. [5] Alternatively, penalised likelihood methods have become popular in survival analysis. These include the use of LASSO, Ridge, and Elastic nets penalties for Proportional Hazards (PH) and Accelerated Failure Time (AFT) models.…”

Prognostic factors and survival disparities in right-sided versus left-sided colon cancer

“…Researchers have used various methodologies to explore risk factors for predicting survival outcomes, including frequentist and Bayesian statistical frameworks. However, frequentist strategies are less efficient when there are high correlations and multiple predictors to consider 25 , 26 . In contrast, Bayesian models offer advantages over frequentist strategies, including demonstrated proficiency in selecting predictors 25 , 27 .…”