Doubly regularized estimation and selection in linear mixed-effects models for high-dimensional longitudinal data

Abstract

The linear mixed-effects model (LMM) is widely used in the analysis of clustered or longitudinal data. This paper aims to address analytic challenges arising from estimation and selection in the application of the LMM to high-dimensional longitudinal data. We develop a doubly regularized approach in the LMM to simultaneously select fixed and random effects. On the theoretical front, we establish large sample properties for the proposed method under the high-dimensional setting, allowing both numbers of fixed effects and random effects to be much larger than the sample size. We present new regularity conditions for the diverging rates, under which the proposed method achieves both estimation and selection consistency. In addition, we propose a new algorithm that solves the related optimization problem effectively so that its computational cost is comparable with that of the Newton-Raphson algorithm for maximum likelihood estimator in the LMM. Through simulation studies we assess performances of the proposed regularized LMM in both aspects of variable selection and estimation. We also illustrate the proposed method by two data analysis examples.

Model selection in linear mixed-effect models

“…Chen et al (2015) demonstrate only the validity of the Oracle property of only sparsity and consistency, but not the asymptotical distribution. Li et al (2018) show the "sparsistency" property which ensures the selection consistency for the true signals of both fixed and random effects; hence, they provide analytical proofs about the validity of consistency and sparsity, but nothing about the distributional form. Pan and Shang (2018b) demonstrate that their procedure fills the consistency and the sparsity properties, without mentioning anything about the asymptotical normality.…”

“…where l R (β (•) ,σ 2 ,ˆ * ) is the REML log-likelihood function related to the model in (32). Li et al (2018) select the two tuning parameter minimizing a variant of BIC, proposed by Wang (2016):…”

“…The zero set (a 2,s+1 , M 2,s+1 ) is discarded from the set of information and the nonzero set replaces a 2 until the iterative penalized REML estimates converge. Li et al (2018) propose a doubly regularized approach for selecting both the fixed and the random effects, in two cases: a) finite dimension of fixed and/or random effects, b) fixed and/or random effects that increase as the sample size goes to infinity. Their approach set = σ 2 I n i and = σ 2 * = σ 2 L L , (Cholesky decomposition) with L a lower triangular matrix containing positive diagonal elements.…”

“…It is worth noting that, all simulations are applied with a moderate number of random effects (for both the full and the true model) and of variance-covariance parameters, except for that of Li et al (2018) and Ahn et al (2012). A large amount of fixed effects occur in the full model of Chen et al (2015), Ghosh and Thoresen (2018) and Rohart et al (2014).…”

“…In the case of high-dimensional settings where the focus is on selection the fixed and the random effects, Li et al (2018) used in their simulations two ways of controlling the tuning parameters: a non-adaptive regularization (NAR), which chooses the tuning parameter from a simple grid of values, and an adaptive regularization (AR), which attributes weights to different penalty parameters. The AR methodology leads to smaller estimation bias for the variance components and to a better control of the false discovery rate.…”