CHull as an alternative to AIC and BIC in the context of mixtures of factor analyzers

Abstract

Mixture analysis is commonly used for clustering objects on the basis of multivariate data. When the data contain a large number of variables, regular mixture analysis may become problematic, because a large number of parameters need to be estimated for each cluster. To tackle this problem, the mixtures-of-factor-analyzers (MFA) model was proposed, which combines clustering with exploratory factor analysis. MFA model selection is rather intricate, as both the number of clusters and the number of underlying factors have to be determined. To this end, the Akaike (AIC) and Bayesian (BIC) information criteria are often used. AIC and BIC try to identify a model that optimally balances model fit and model complexity. In this article, the CHull (Ceulemans & Kiers, 2006) method, which also balances model fit and complexity, is presented as an interesting alternative model selection strategy for MFA. In an extensive simulation study, the performances of AIC, BIC, and CHull were compared. AIC performs poorly and systematically selects overly complex models, whereas BIC performs slightly better than CHull when considering the best model only. However, when taking model selection uncertainty into account by looking at the first three models retained, CHull outperforms BIC. This especially holds in more complex, and thus more realistic, situations (e.g., more clusters, factors, noise in the data, and overlap among clusters).Keywords Mixture analysis . Model selection . AIC . BIC . CHullIn the behavioral sciences, researchers often cluster multivariate (i.e., object-by-variable) data in order to capture the heterogeneity that is present in the population. The resulting clusters can differ with regard to their level and/ or covariance structure. A first example pertains to the case in which a number of children are scored on certain psychopathological symptoms. The aim then is to discern different groups and to describe the differences between the groups in terms of the strength of the symptoms and/ or of their linear covariation. A second example is a consumer psychologist who wants to identify different groups of consumers on the basis of their appraisals of a wide range of food products.A commonly used clustering method is mixture analysis (McLachlan & Peel, 2000). In this method, each cluster is described by a different multivariate distribution, and every object belongs to each cluster with a particular probability. As a result, the full data follow a mixture of multivariate distributions. In practice, because of their computational simplicity, multivariate normal distributions are often assumed (McLachlan, Peel, & Bean, 2003), implying that each cluster is characterized by a mean vector and a covariance matrix.When the number of variables increases, such a mixture of multivariate normals may become problematic, in that a large number of variance and covariance parameters need to be estimated for each cluster [i.e., for J variables, J(J + 1)/2 variances and covariances need to be determined]. This problem is aggr...

Genetic and Environmental Structure of DSM-IV Criteria for Antisocial Personality Disorder: A Twin Study

“…Thus, the moderate attrition effects (Tambs et al 2009) appear to be an unlikely source of bias for our main findings. As a general limitation applicable to all related studies, optimal information criterion for purposes of model selection is still a debated topic (Markon and Krueger 2004; Nylund et al 2007; Vrieze 2012; Bulteel et al 2013), and even in the cases where BIC indicated very strong support for a model, SABIC did not always do so (Table 2 and Table S3). …”

Investigating phenotypes of emotional eating based on weight categories: A latent profile analysis

“…Smaller values for information criteria indicate better model fit (Berlin, Williams, & Parra, ). Further, as it has been found that AIC tends to overestimate the true number of classes (Bulteel, Wilderjans, Tuerlinckx, & Ceulemans, ), BIC has been generally used over AIC. Entropy, ranging from 0 to 1 (higher values indicate better fit), indicates how accurate the model is in classifying individuals into classes (Berlin et al, ), and was therefore also taken into consideration.…”

DeCon: A tool to detect emotional concordance in multivariate time series data of emotional responding

“…DeCon sets the number of components used to determine “outlyingness” by analyzing the data in each of the first ten windows and applying an automated scree test (Ceulemans & Kiers, 2006; Wilderjans, Ceulemans, & Meers, 2013) to each window. The performance of this automated scree test procedure was extensively studied for multiple models using simulations, yielding consistently good results (e.g., Bulteel, Wilderjans, Tuerlinckx, & Ceulemans, 2013; Ceulemans & Kiers, 2009; Ceulemans, Timmerman, & Kiers, 2011; Schepers, Ceulemans, & Van Mechelen, 2008). Still, some uncertainty in the selection of the number of components may remain due to sampling fluctuations.…”