A Simulation Study for the AIC and Likelihood Cross-validation: The Case of Exponential Versus Weibull Distributions

Abstract

Various methods are available for choosing statistical models. It is difficult to know which model selection criterion is the best for specific data. This paper discusses a method for choosing the model selection criterion based on the characteristics of the data and models. As an example, we examined the choice between AIC and likelihood cross-validation as the model selection criterion with the exponential distribution and Weibull distribution as candidate models. First, we examined the characteristics of AIC and likelihood cross-validation using data generated from an exponential distribution or Weibull distribution; AIC and likelihood cross-validation show substantially different natures. Next, from the results of the numerical simulations, we propose an intuitive method for deciding whether to use AIC or likelihood cross-validation.

Testing the Fairness of a Coin by Akaike's Information Criterion

“…Hence, when we do not know whether the available data satisfy such conditions, AIC • (−0.5) might not be a good approximation of the expected log-likelihood. In fact, when we handle a simple problem that chooses between an exponential distribution and Weibull distribution, AIC • (−0.5) cannot be used as an approximation of the expected log-likelihood in most situations ( [3]). Nevertheless, we rarely ascertain that the data at hand satisfy such conditions when we use AIC in practice.…”