An Empirical Comparison of Information-Theoretic Selection Criteria for Multivariate Behavior Genetic Models

Abstract

Information theory provides an attractive basis for statistical inference and model selection. However, little is known about the relative performance of different information-theoretic criteria in covariance structure modeling, especially in behavioral genetic contexts. To explore these issues, information-theoretic fit criteria were compared with regard to their ability to discriminate between multivariate behavioral genetic models under various model, distribution, and sample size conditions. Results indicate that performance depends on sample size, model complexity, and distributional specification. The Bayesian Information Criterion (BIC) is more robust to distributional misspecification than Akaike's Information Criterion (AIC) under certain conditions, and outperforms AIC in larger samples and when comparing more complex models. An approximation to the Minimum Description Length (MDL; Rissanen, J. (1996). IEEE Transactions on Information Theory 42:40-47, Rissanen, J. (2001). IEEE Transactions on Information Theory 47:1712-1717) criterion, involving the empirical Fisher information matrix, exhibits variable patterns of performance due to the complexity of estimating Fisher information matrices. Results indicate that a relatively new information-theoretic criterion, Draper's Information Criterion (DIC; Draper, 1995), which shares features of the Bayesian and MDL criteria, performs similarly to or better than BIC. Results emphasize the importance of further research into theory and computation of information-theoretic criteria.

Weighted Average Information Criterion for Selection of an Asymmetric Price Relationship

“…This is consistent with previous studies (Acquah 2013;Tan & Biswas 2012;Al-Marshadi, 2009;Markon & Krueger, 2004;Bozdogan, 1987;Atkinson, 1980) Recovery rates of Weighted Average Information Criteria strongly depended on sample size for the true data generating process (DGP). It increased from 37.7 percent to 100.0 percent when the sample size was increased from 50 to 500.…”

A Longitudinal Study on Genetic and Environmental Influences on Leisure Time Physical Activity in the Finnish Twin Cohort

“…After dropping the weakest parameters, the model with the best fit to the data according to the DIC contained additive genetic (A) and specific environmental (E) influences with equal estimates in men and in women. Because the model with gender differences may also provide the best fit according to the AIC and BIC, it is preferable to report the results from both models, with and without gender differences in the parameter estimates (Markon, 2004).…”

“…Initially, smaller AIC, BIC or DIC values normally indicate a better fit to the data. When a discrepancy was noticed between these criteria, preference was given to the models achieving best fit with the BIC and DIC, as they are thought to perform better than the other criteria in more complex models with relatively large sample sizes (Markon, 2004). In addition, because the sample was composed of men and women, the significance of potential gender differences in the estimates were tested by comparing model fitting statistics from a model that constrains the A, C, and E parameters to be equal for men and women with models where these parameters are allowed to differ by gender.…”

A genetically informative developmental study of the relationship between conduct disorder and peer deviance in males

“…We used the Bayesian Information Criterion (BIC; Schwarz, 1978), which performs well with such complex models (Markon & Krueger, 2004). By minimizing the BIC, we sought to optimize the balance of explanatory power and parsimony.…”