Computation for Latent Variable Model Estimation: A Unified Stochastic Proximal Framework

Abstract

Latent variable models have been playing a central role in psychometrics and related fields. In many modern applications, the inference based on latent variable models involves one or several of the following features: (1) the presence of many latent variables, (2) the observed and latent variables being continuous, discrete, or a combination of both, (3) constraints on parameters, and (4) penalties on parameters to impose model parsimony. The estimation often involves maximizing an objective function based on a marginal likelihood/pseudo-likelihood, possibly with constraints and/or penalties on parameters. Solving this optimization problem is highly non-trivial, due to the complexities brought by the features mentioned above. Although several efficient algorithms have been proposed, there lacks a unified computational framework that takes all these features into account. In this paper, we fill the gap. Specifically, we provide a unified formulation for the optimization problem and then propose a quasi-Newton stochastic proximal algorithm. Theoretical properties of the proposed algorithms are established. The computational efficiency and robustness are shown by simulation studies under various settings for latent variable model estimation.

Efficient Metropolis-Hastings Robbins-Monro Algorithm for High-Dimensional Diagnostic Classification Models

“…The second stage continues with w = 0, where the average of the iterates are used as starting values for the next stage. For the final stage, the iterative averaging procedure with 12<w<1 was shown as promising for finite-sample problems (Polyak & Juditsky, 1992; Ruppert, 1991), which has also been successfully applied to the parameter estimation of latent variable models (Zhang & Chen, 2020). Specifically, trueθ¯t=(t−s)−1∑h=s+1ttrueθ^h shows this, where each of the trueθ^h is computed as in the typical RM algorithm and s denotes the “burn-in” length.…”

Latent variable selection in multidimensional item response theory models using the expectation model selection algorithm

“…(2016) applied the EM algorithm (Dempster et al., 1977) combined with the coordinate descent algorithm (Friedman, Hastie, & Tibshirani, 2010), and they chose the regularization parameter with the smallest BIC value. Zhang and Chen (2021) proposed a quasi‐Newton stochastic proximal algorithm to optimize the L 1 penalized log‐likelihood, and they proved that their algorithm converges to a stationary point of the L 1 penalized log‐likelihood. Moreover, Zhang and Chen (2021) did not assume constraint (ii) on A as described in Section 2.1, and they stated that the rotational indeterminacy issue is resolved in the L 1 regularized estimator.…”

“…Zhang and Chen (2021) proposed a quasi‐Newton stochastic proximal algorithm to optimize the L 1 penalized log‐likelihood, and they proved that their algorithm converges to a stationary point of the L 1 penalized log‐likelihood. Moreover, Zhang and Chen (2021) did not assume constraint (ii) on A as described in Section 2.1, and they stated that the rotational indeterminacy issue is resolved in the L 1 regularized estimator. Unfortunately, this is not true in the L 0 regularized estimator in this paper.…”

“…To make a fair comparison between EMS and EML1, we also used glment for minimizing in the MS‐step of the EMS. Though the L 1 regularized estimator can also be solved by the quasi‐Newton stochastic proximal algorithm (Zhang & Chen, 2021), we did not compare it with our method because their code was not available. All R code is available at https://github.com/Laixu3107/EMS-for-MIRT.…”

“…Since the L 1 regularized log‐likelihood involves the integration of latent variables, the EM algorithm (Dempster, Laird, & Rubin, 1977) is applied by treating latent traits as the missing data. Zhang and Chen (2021) proposed a quasi‐Newton stochastic proximal algorithm for the estimation of a wide range of latent variable models including high‐dimensional IFA models. Another approach for obtaining a sparse solution is given in Battauz (2020), where a fused lasso penalty is used to group response categories in the nominal response models.…”