Maximizing Generalized Linear Mixed Model Likelihoods With an Automated Monte Carlo EM Algorithm

Abstract

Two new implementations of the EM algorithm are proposed for maximum likelihood ®tting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more ef®cient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Speci®cally, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more ef®cient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.

A mechanistic nonlinear model for censored and mismeasured covariates in longitudinal models, with application in AIDS studies

“…For the MCEM method for the joint covariate and response model, we start with k 0 = 100 Monte Carlo samples and increase the Monte Carlo sample size 23 as the number of iteration t increases: k t+1 = k t + k t ∕c with c = 4. Convergence of the EM algorithm was considered to be achieved when the maximum percentage change of all estimates was less than 0.1% in 2 consecutive iterations.…”

A comparative study of approximation methods for maximum likelihood estimation in generalized linear mixed models (GLMM)

“…In general, the approximation method for parameter estimation in GLMM consists of: (a) simplification of problem analytically (using Laplace approximation to integrate the likelihood function), such as Penalized Quasi Likelihood [1] and Hierarchical Generalized Linear models or HGLM [2] (b) computer intensive methods, such as Monte Carlo EM algorithm [3], Markov Chain Monte Carlo or MCMC [4] and Gauss-Hermite quadrature or GHQ [5]. The Laplace approximation approximates the integrand, PQL approximates the data and AGQ approximates the integral.…”

Alternating imputation posterior estimation of models with crossed random effects

“…In the E-step, a Gibbs sampler is used to sample the random effects from their posterior distributions and the M-step is the estimation of a generalized linear mixed model. Booth and Hobert (1999) implemented MCEM employing importance sampling in the E-step and Vaida and Meng (2005) employing a slice sampler to fit generalized linear models with crossed random effects. This algorithm is also computationally expensive requiring many draws of the random effects in the E-step to achieve a sufficiently small Monte Carlo error close to convergence.…”