Order Selection and Sparsity in Latent Variable Models via the Ordered Factor LASSO

Abstract

Generalized linear latent variable models (GLLVMs) offer a general framework for flexibly analyzing data involving multiple responses. When fitting such models, two of the major challenges are selecting the order, that is, the number of factors, and an appropriate structure for the loading matrix, typically a sparse structure. Motivated by the application of GLLVMs to study marine species assemblages in the Southern Ocean, we propose the Ordered Factor LASSO or OFAL penalty for order selection and achieving sparsity in GLLVMs. The OFAL penalty is the first penalty developed specifically for order selection in latent variable models, and achieves this by using a hierarchically structured group LASSO type penalty to shrink entire columns of the loading matrix to zero, while ensuring that non-zero loadings are concentrated on the lower-order factors. Simultaneously, individual element sparsity is achieved through the use of an adaptive LASSO. In conjunction with using an information criterion which promotes aggressive shrinkage, simulation shows that the OFAL penalty performs strongly compared with standard methods and penalties for order selection, achieving sparsity, and prediction in GLLVMs. Applying the OFAL penalty to the Southern Ocean marine species dataset suggests the available environmental predictors explain roughly half of the total covariation between species, thus leading to a smaller number of latent variables and increased sparsity in the loading matrix compared to a model without any covariates.

Assuming independence in spatial latent variable models: Consequences and implications of misspecification

“…Finally, in our simulations we assumed that the true number of latent variables is known, when in practice this also has been chosen (e.g., Hui et al, 2018). How misspecifying the latent variable correlation structures affects the number of latent variables chosen is an avenue of future research to pursue, along with the more general topic of how misspecification of the latent variable correlation structure affects variable selection as well as other aspects of inference for GLLVMs as a whole (see Hoeting et al, 2006;Xu et al, 2015, for Simulation results for empirical bias (left column), root mean squared error (middle column), and coverage probability (right column) of regression coefficients β j , for normal response GLLVMs.…”

Efficient estimation of generalized linear latent variable models

“…To date we have used the software to fit a dataset of size 174 × 985 in 61 minutes. In future work, we plan to generalize GLLVMs, as well as the package, so that it can handle spatial and or temporal correlation inherent in the data, as well as offer some data-driven forms of order and variable selection (see for example [43]).…”

Model-based ordination for species with unequal niche widths

“…Determining if species exhibit fully quadratic curves in response to ecological gradients, whether tolerances are common for all species per ecological gradient, or if the equal tolerances assumption is suited for a dataset, comes down to a problem of model selection for the quadratic GLLVM. To that end, future research can further investigate approaches such as regularization (e.g., possibly extending the approach of Hui et al 2018), hypothesis testing, or the use of confidence intervals of the quadratic coefficients. Similar to DCA, the quadratic GLLVM provides estimates of gradient length.…”