A new method is proposed that extends the use of regularization in both lasso and ridge regression to structural equation models. The method is termed regularized structural equation modeling (RegSEM). RegSEM penalizes specific parameters in structural equation models, with the goal of creating easier to understand and simpler models. Although regularization has gained wide adoption in regression, very little has transferred to models with latent variables. By adding penalties to specific parameters in a structural equation model, researchers have a high level of flexibility in reducing model complexity, overcoming poor fitting models, and the creation of models that are more likely to generalize to new samples. The proposed method was evaluated through a simulation study, two illustrative examples involving a measurement model, and one empirical example involving the structural part of the model to demonstrate RegSEM's utility.
Keywordsfactor analysis; lasso; penalization; regularization; ridge; shrinkage; structural equation modeling The desire for model simplicity in exploratory factor analysis (EFA) has led to the longstanding goal of achieving simple structure. By simple structure, we are broadly referring to a pattern of factor loadings where each indicator is influenced by a minimal number of latent factors, preferably one. Multiple researchers have set different criteria for achieving simple structure, with the most notable being Thurstone (1935Thurstone ( , 1947. However, the desire for simplicity in model structure comes by many names, including the aforementioned simple structure (Thurstone, 1935), variable complexity (Browne, 2001, parsimony (Marsh & Hau, 1996;Raykov & Marcoulides, 1999), "sparse loadings" in the context of principal components analysis (PCA; Zou, Hastie, & Tibshirani, 2006), and finally, sparsistency, denoting that all parameters in a sparse model that are zero are correctly estimated as zero with probability tending to one (Lam & Fan, 2009). Across multiple fields and different terminology, the goal is roughly the same: to accurately and efficiently estimate a model that is parsimonious in allowing users to easily interpret the model's representation of reality.However, simpler is not always better. Overconstraining models in an attempt to make interpretation easier, can lead to unacceptable levels of misfit and biased parameter estimates Correspondence should be addressed to Ross Jacobucci, Department of Psychology, University of Southern California, SGM 501, 3620 South McClintock Avenue, Los Angeles, CA 90089, USA. jacobucc@usc.edu.
HHS Public Access
Author Manuscript Author ManuscriptAuthor ManuscriptAuthor Manuscript (Hsu, Troncoso Skidmore, Li, & Thompson, 2014;Muthén & Asparouhov, 2012). In a confirmatory factor analysis (CFA), biased parameter estimates manifest as inflated covariances between latent factors caused by not allowing cross-loadings or residual covariances. It is not that the goal of simple structure is a poor choice, but that blind adherence to this st...