On the prevalence of uninformative parameters in statistical models applying model selection in applied ecology

Abstract

Research in applied ecology provides scientific evidence to guide conservation policy and management. Applied ecology is becoming increasingly quantitative and model selection via information criteria has become a common statistical modeling approach. Unfortunately, parameters that contain little to no useful information are commonly presented and interpreted as important in applied ecology. I review the concept of an uninformative parameter in model selection using information criteria and perform a literature review to measure the prevalence of uninformative parameters in model selection studies applying Akaike’s Information Criterion (AIC) in 2014 in four of the top journals in applied ecology (Biological Conservation, Conservation Biology, Ecological Applications, Journal of Applied Ecology). Twenty-one percent of studies I reviewed applied AIC metrics. Many (31.5%) of the studies applying AIC metrics in the four applied ecology journals I reviewed had or were very likely to have uninformative parameters in a model set. In addition, more than 40% of studies reviewed had insufficient information to assess the presence or absence of uninformative parameters in a model set. Given the prevalence of studies likely to have uninformative parameters or with insufficient information to assess parameter status (71.5%), I surmise that much of the policy recommendations based on applied ecology research may not be supported by the data analysis. I provide four warning signals and a decision tree to assist authors, reviewers, and editors to screen for uninformative parameters in studies applying model selection with information criteria. In the end, careful thinking at every step of the scientific process and greater reporting standards are required to detect uninformative parameters in studies adopting an information criteria approach.

Trophic rewilding establishes a landscape of fear: Tasmanian devil introduction increases risk‐sensitive foraging in a key prey species

“…Some models showed strong signs of containing a 'pretending variable' (sensu Anderson 2007), otherwise known as an uninformative parameter (Leroux 2019). the log-likelihood) and increases the AIC value by approximately the penalty of two (Anderson 2007, Leroux 2019). the log-likelihood) and increases the AIC value by approximately the penalty of two (Anderson 2007, Leroux 2019).…”

“…These variables can be identified when the addition of a variable to a simpler nested model does not improve model fit (i.e. In such cases, we excluded models containing a pretending variable, as recommended by Anderson (2007) and Leroux (2019). In such cases, we excluded models containing a pretending variable, as recommended by Anderson (2007) and Leroux (2019).…”

Predicting species distributions: unifying model selection and scale optimization for multi‐scale occupancy models

“…We demonstrate empirically what is often taken for granted in applied ecological analyses, that selecting a model based on out‐of‐sample prediction is different than balancing fit and complexity using only within‐sample data and pre‐specified penalty terms. Information criteria attempt to balance fit and complexity, but often select over‐parameterized models (Arnold , Barker and Link , Leroux ) and at best only approximate a model's predictive ability. Information criteria are often used for their computational simplicity, avoiding the need for a testing data set or the burden of cross‐validation, but potentially compromising the assessment of predictive ability by use of within‐sample approximation (Hooten and Hobbs ).…”

A native apex predator limits an invasive mesopredator and protects native prey: Tasmanian devils protecting bandicoots from cats

“…We created all simpler combinations of the most complex model and selected the best‐performing models using AICc (Burnham & Anderson ). We assessed whether high‐ranking models contained uninformative parameters, which are often present when comparing nested models, simply because the inclusion of an uninformative parameter receives a penalty of 2 AIC points (Anderson ; Leroux ). Uninformative parameters can be identified when their addition to a simpler nested model causes little improvement in the log‐likelihood and when confidence intervals for the parameter estimate span zero (Anderson ; Leroux ).…”

“…Uninformative parameters can be identified when their addition to a simpler nested model causes little improvement in the log‐likelihood and when confidence intervals for the parameter estimate span zero (Anderson ; Leroux ). In such cases, we omitted the model (Leroux ). We predicted abundance and standard errors for each of the 28 study sites, either from the best model when there was a clear winning model, or a model‐averaged prediction when competing models were within 7ΔAICc (Burnham et al ).…”