Simple Formula for Calculating Bias‐corrected AIC in Generalized Linear Models

Abstract

In real-data analysis, deciding the best subset of variables in regression models is an important problem. Akaike's information criterion (AIC) is often used in order to select variables in many fields. When the sample size is not so large, the AIC has a non-negligible bias that will detrimentally affect variable selection. The present paper considers a bias correction of AIC for selecting variables in the generalized linear model (GLM). The GLM can express a number of statistical models by changing the distribution and the link function, such as the normal linear regression model, the logistic regression model, and the probit model, which are currently commonly used in a number of applied fields. In the present study, we obtain a simple expression for a biascorrected AIC (corrected AIC, or CAIC) in GLMs. Furthermore, we provide an 'R' code based on our formula. A numerical study reveals that the CAIC has better performance than the AIC for variable selection.

A comparison of model choice strategies for logistic regression

“…AICc as BICc below (McQuarrie, 1999) has been derived for the Gaussian regression model. There is a corrected AIC for generalized linear models (Imori et al, 2014) but it was found too slow for the Monte Carlo simulations of this study.…”

Asymptotic Cumulants of Some Information Criteria

“…Since in practice information criteria are used typically for model selection, simulations using 1) the n −1 AIC (= n −1 AIC ML ), 2) the bias corrected n −1 AIC i.e., n −1 AIC − n −2ĉ 1 denoted by n −1 CAIC (= n −1 CAIC ML ), and 3) n −1 AIC W with the Jeffreys prior for selecting regressors are carried out in this section when a regression model holds under canonical parametrization. Regularity conditions in regression with fixed covariates are assumed to be satisfied as in Fahrmeir and Kaufmann (1985), Boik (2008), and Imori, Yanagihara and Wakaki (2014). Four types of regression: logistic, Poisson, negative binomial and gamma regression are used, where a canonical parameter has a form of the linear combination of p regressors including an intercept when it is used.…”

“…Bias corrections of the AICs in logistic and Poisson regression are given by Yanagihara, Sekiguchi and Fujikoshi (2003) and Kamo, Yanagihara and Satoh (2013), respectively by different methods and expressions from those in this section. Imori, Yanagihara and Wakaki (2014) gave the corresponding results in the generalized linear model (GLM) for the natural location parameter when the scale parameter in the GLM is known. Since the unified result of bias correction for the n −1 AIC under canonical parametrization in the exponential family was derived earlier (see Corollary 2; (A3.1) and (A3.2) in Ogasawara, 2016, Appendix), e.g., logistic regression and Poisson regression are also dealt with as special cases in this section.…”

“…Similarly, the result of bias correction of n −1 AIC in gamma regression with an unknown shape parameter is new. The illustrations of n −1 AIC and n −1 CAIC (n −1 CAIC * ) in negative binomial regression are new though the result when the shape parameter is known is seen as a special case of Imori, Yanagihara and Wakaki (2014).…”

“…Interval estimation of the corresponding population quantities can also be done similarly. While the above methods of testing and estimation are for general models, the results for special models are available for the higher-order bias correction from Sugiura (1978), Yanagihara, Sekiguchi and Fujikoshi (2003), Kamo, Yanagihara and Satoh (2013), Imori, Yanagihara and Wakaki (2014), and the asymptotic cumulants for standardized estimators from Yanagihara and Ohmoto (2005), among others.…”