Variable selection for mixed data clustering: a model-based approach

Abstract

We propose two approaches for selecting variables in latent class analysis (i.e., mixture model assuming within component independence), which is the common model-based clustering method for mixed data. The first approach consists in optimizing the BIC with a modified version of the EM algorithm. This approach simultaneously performs both model selection and parameter inference. The second approach consists in maximizing the MICL, which considers the clustering task, with an algorithm of alternate optimization. This approach performs model selection without requiring the maximum likelihood estimates for model comparison, then parameter inference is done for the unique selected model. Thus, the benefits of both approaches is to avoid the computation of the maximum likelihood estimates for each model comparison. Moreover, they also avoid the use of the standard algorithms for variable selection which are often suboptimal (e.g. stepwise method) and computationally expensive. The case of data with missing values is also discussed. The interest of both proposed criteria is shown on simulated and real data.

Variable selection methods for model-based clustering

“…The authors suggest the use of slope heuristics (see Baudry, Maugis and Michel, 2012, for an overview) to calibrate the value of κ. Marbac and Sedki (2017c) extend the approach of Marbac and Sedki (2017a) (see Section 4.3) to clustering and variable selection of data of mixed type, with data on only categorical variables as a particular case. The method is based again on the MICL criterion and does not require multiple calls of the EM algorithm.…”

“…The R packages for variable selection for latent class analysis are ClustMMDD (Toussile, 2016), LCAvarsel and VarSelLCM (Marbac and Sedki, 2017b). In particular, we note that VarSelLCM implements a more general framework for clustering and variable selection of data of mixed type (Marbac and Sedki, 2017c). Table 4 lists the packages, with information regarding the type and the method; all implement a model selection approach.…”

“…Variable selection for model-based clustering with other distributions and/or of data of different nature is still an open research area. The topic has only recently started to attract attention: Wallace et al (2017) propose a framework for clustering and variable selection with the multivariate skew-Normal distribution, while Marbac and Sedki (2017c) define a method for data of mixed types. There is certainly a wide scope for further investigations and developments in this direction.…”

“…Bartolucci, Montanari and Pandolfi (2016) use similar arguments for clustering categorical data via the latent class analysis model. The same MAR assumption is also considered in Marbac and Sedki (2017c), where the MICL criterion used for variable selection is computed considering only the observed entries. In application to data related to the quality-of-life of elderly patients hosted in nursing homes, Bartolucci, Montanari and Pandolfi (2017) move away from the MAR assumption and suggest to add an extra category corresponding to the missing responses, circumventing the assumption of ignorability of the missing data.…”

A tractable multi-partitions clustering

“…In order to deal with large numbers of variables, we propose an extension of the approaches proposed by Marbac and Sedki (2017b) and Marbac and Sedki (2017a), in the framework of variable selection in clustering. The main idea is to use a more constrained model to be able to easily perform model selection.…”