CHull: A generic convex-hull-based model selection method

Abstract

When analyzing data, researchers are often confronted with a model selection problem (e.g., determining the number of components/factors in principal components analysis [PCA]/factor analysis or identifying the most important predictors in a regression analysis). To tackle such a problem, researchers may apply some objective procedure, like parallel analysis in PCA/factor analysis or stepwise selection methods in regression analysis. A drawback of these procedures is that they can only be applied to the model selection problem at hand. An interesting alternative is the CHull model selection procedure, which was originally developed for multiway analysis (e.g., multimode partitioning). However, the key idea behind the CHull procedure-identifying a model that optimally balances model goodness of fit/misfit and model complexity-is quite generic. Therefore, the procedure may also be used when applying many other analysis techniques. The aim of this article is twofold. First, we demonstrate the wide applicability of the CHull method by showing how it can be used to solve various model selection problems in the context of PCA, reduced Kmeans, best-subset regression, and partial least squares regression. Moreover, a comparison of CHull with standard model selection methods for these problems is performed. Second, we present the CHULL software, which may be downloaded from http://ppw.kuleuven.be/okp/software/CHULL/, to assist the user in applying the CHull procedure.Keywords Model selection . CHULL . Graphical user interface . PCA . Regression . PLS When analyzing data, researchers very often face a (complex) model selection problem. Take, as a first example, a clinical psychologist who wants to study the dimensionality of a particular psychological construct, such as alexithymia (i.e., having difficulties distinguishing between and expressing emotions). To this end, the researcher administers to a sample of subjects a questionnaire that measures the construct in question. Next, the researcher may examine the internal structure of the questionnaire and the dimensionality of the underlying construct by performing a principal components analysis (PCA) or an exploratory factor analysis (EFA) on the collected data. In doing so, the researcher needs to determine the optimal number of components or factors, and thus has to solve a model selection problem. As a second example, consider an economist who wants to assess which "factors" affect the selling price of a house in a particular neighborhood. To study this, the researcher may collect data (e.g., price and size of the house or the number of rooms) about a sample of houses from the neighborhood under study and may perform a regression analysis. In this case, again a model selection problem arises, which consists of selecting the (sub)set of predictors (optionally also including interactions between the predictors) that optimally predicts the selling price of a house.In general, model selection boils down to selecting, out of a set of models, one that yields a good...

Thinking about assessment: Further evidence of the validity of the Movie for the Assessment of Social Cognition as a measure of mentalistic abilities.

“…We performed dimensionality analyses of the MASC items based on differing criteria; in particular, the minimum average partial statistic (MAP; Zwick & Velicer, 1986), quasi-inferential parallel analysis (Buja & Eyuboglu, 1992), and Hull's method (Lorenzo-Seva, Timmerman, & Kiers, 2011;Wilderjans, Ceulemans, & Meers, 2013) were used for determining the number of latent dimension underlying the tetrachoric correlation matrix of the MASC items.…”

Maximal Interaction Two-Mode Clustering

“…In the framework of two-mode partitioning problems, various procedures for choosing the appropriate numbers of clusters P and Q have been proposed in the literature (see, e.g., Ceulemans and Kiers, 2006;Schepers, Ceulemans, and Van Mechelen, 2008;and Wilderjans, Ceulemans, and Meers, 2013). Instead of presenting a full analysis of our data, we refer, for illustration purposes, only to the optimal biclusterings with P +Q ≤ 5.…”

Reliability and validity of the Italian translation of the UPPS-P Impulsive Behavior Scale in a sample of consecutively admitted psychotherapy patients

“…In order to examine the dimensions underlying the UPPS-P item polychoric correlation matrix, we used quasi-inferential parallel analysis (Buja and Eyuboglu, 1992) and Hull's method (Wilderjans et al, 2013). Following Booth and Huges' (2014) suggestions, we used both weighted least square mean and variance corrected (WLSMV) confirmatory factor analysis (CFA) and exploratory structural equation modeling (ESEM; Marsh et al, 2014) to examine the a priori five-factor model of the UPPS-P items.…”