Asymptotic post-selection inference for the Akaike information criterion

Charkhi, Ali; Claeskens, Gerda

doi:10.1093/biomet/asy018

Cited by 36 publications

(48 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is an active area of research (Claeskens and Hjort 2008, Berk et al. 2013, Charkhi and Claeskens 2018, Kabaila and Wijethunga 2019). For predictive inference, the post‐selection confidence interval of the mean response can be estimated using techniques based on model averaging (Hjort and Claeskens 2003, Efron 2014).…”

Section: Discussionmentioning

confidence: 99%

Parsimonious model selection using information theory: a modified selection rule

Yates

Richards

Brook

2021

Ecology

View full text Add to dashboard Cite

Information‐theoretic approaches to model selection, such as Akaike's information criterion (AIC) and cross validation, provide a rigorous framework to select among candidate hypotheses in ecology, yet the persistent concern of overfitting undermines the interpretation of inferred processes. A common misconception is that overfitting is due to the choice of criterion or model score, despite research demonstrating that selection uncertainty associated with score estimation is the predominant influence. Here we introduce a novel selection rule that identifies a parsimonious model by directly accounting for estimation uncertainty, while still retaining an information‐theoretic interpretation. The new rule, which is a modification of the existing one‐standard‐error rule, mitigates overfitting and reduces the likelihood that spurious effects will be included in the selected model, thereby improving its inferential properties. We present the rule and illustrative examples in the context of maximum‐likelihood estimation and Kullback‐Leibler discrepancy, although the rule is applicable in a more general setting, including Bayesian model selection and other types of discrepancy.

show abstract

Section: Discussionmentioning

confidence: 99%

Parsimonious model selection using information theory: a modified selection rule

Yates

Richards

Brook

2021

Ecology

View full text Add to dashboard Cite

show abstract

“…By using information about the specifics of the selection method such inference methods result in narrower confidence intervals as compared to the Berk et al (2013) method. The effect of increasing the number of models results in getting larger confidence intervals (see Charkhi and Claeskens, 2018). Valid inference after selection is currently investigated for several model selection methods.…”

Section: Discussionmentioning

confidence: 99%

Model Selection via Focused Information Criteria for Complex Data in Ecology and Evolution

2019

Self Cite

View full text Add to dashboard Cite

Datasets encountered when examining deeper issues in ecology and evolution are often complex. This calls for careful strategies for both model building, model selection, and model averaging. Our paper aims at motivating, exhibiting, and further developing focused model selection criteria. In contexts involving precisely formulated interest parameters, these versions of FIC, the focused information criterion, typically lead to better final precision for the most salient estimates, confidence intervals, etc. as compared to estimators obtained from other selection methods. Our methods are illustrated with real case studies in ecology; one related to bird species abundance and another to the decline in body condition for the Antarctic minke whale.

show abstract

“…In Cunen, Walløe and Hjort (2020), we bypassed this problem by splitting the data into two parts, one for model selection and the second for inference. This is naturally a conservative solution, but there are more sophisticated approaches in the general model selection literature, see Berk et al (2013); Tibshirani et al (2016); Charkhi and Claeskens (2018). Naturally, and perhaps trivially, one can avoid the issues of post-selection, and the complexities of model selection in general, by choosing to use the wide model and not do model selection at all, see for instance Ver Hoef and Boveng (2015).…”

Section: Descriptionmentioning

confidence: 99%

Focused model selection for linear mixed models with an application to whale ecology

Cunen¹,

Walløe²,

Hjort³

2020

Ann. Appl. Stat.

View full text Add to dashboard Cite

A central point of disagreement, in certain long-standing discussions about a particular whaling dataset in the Scientific Committee of the International Whaling Commission, has directly involved model selection issues for linear mixed effect models. The biological question under discussion is associated with a clearly defined parameter of primary interest, i.e. a focus parameter, which makes model selection with the Focused Information Criterion (FIC) more appropriate than other selection methods. Since the existing FIC methodology has not covered the case of linear mixed effects models, this article sets up the required framework and develops the necessary formulae for the relevant FIC. Our new criterion requires the asymptotic distribution of estimators derived for a given candidate linear mixed model, but with behaviour examined under a wider linear mixed model. These results, needed here to build our FIC, also have independent interest.

show abstract

Asymptotic post-selection inference for the Akaike information criterion

Cited by 36 publications

References 35 publications

Parsimonious model selection using information theory: a modified selection rule

Parsimonious model selection using information theory: a modified selection rule

Model Selection via Focused Information Criteria for Complex Data in Ecology and Evolution

Focused model selection for linear mixed models with an application to whale ecology

Contact Info

Product

Resources

About