Performance of variable selection methods for assessing the health effects of correlated exposures in case–control studies

Abstract

In this extensive simulation study with multicollinear data, we found that most variable selection methods consistently outperformed conventional approaches, and demonstrated how performance is influenced by the structure of the data and underlying model.

Performance of variable and function selection methods for estimating the nonlinear health effects of correlated chemical mixtures: A simulation study

“…Standard implementations of lasso assume linearity, and groups of terms associated with an exposure cannot be selected or dropped simultaneously. When exposure‐response relationships are linear, lasso has achieved comparable or superior performance in terms of sensitivity and specificity to competing variable selection methods 6,9,10 . Lasso produces a family of solutions, parameterized by a tuning parameter.…”

Modern statistics, multiple testing and wishful thinking

“…Hypothesis testing is at the very base of understanding, along with modelling and parameter estimation. An article in this issue by Lenters et al 1 uses simulation to address some questions which should be well understood in the epidemiology community, but sadly are not.…”

“…In contrast, Lenters et al 1 attempt to compare different methods which differ in both specificity (measured as 1−FDP) and sensitivity. They propose identifying an overall best method by a combination of graphical presentations and the ‘F-measure’, which is a weighted hyperbolic average of sensitivity and specificity.…”

“…As presented, only one of the models discussed in Lenters  et al 1 is calibrated for correct type 1 error performance (Bayesian LASSO) and is the only one that is fit for use as parametrised. This use of methods with default parameter settings may encourage researchers to focus excessively on the publications they might gain through more sensitivity without concern for the loss of credibility everyone suffers through the loss of specificity.…”

“…Understanding our tools is a primary responsibility in this, and we should all be carrying out simple simulations to see if we are likely to be finding associations more often than we should. I would therefore recommend reading Lenters et al ’ s 1 paper not to simply choose a canned method of analysis, but to see how seriously we can be led astray by default parameter values.…”