Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes

Abstract

BackgroundLogistic random effects models are a popular tool to analyze multilevel also called hierarchical data with a binary or ordinal outcome. Here, we aim to compare different statistical software implementations of these models.MethodsWe used individual patient data from 8509 patients in 231 centers with moderate and severe Traumatic Brain Injury (TBI) enrolled in eight Randomized Controlled Trials (RCTs) and three observational studies. We fitted logistic random effects regression models with the 5-point Glasgow Outcome Scale (GOS) as outcome, both dichotomized as well as ordinal, with center and/or trial as random effects, and as covariates age, motor score, pupil reactivity or trial. We then compared the implementations of frequentist and Bayesian methods to estimate the fixed and random effects. Frequentist approaches included R (lme4), Stata (GLLAMM), SAS (GLIMMIX and NLMIXED), MLwiN ([R]IGLS) and MIXOR, Bayesian approaches included WinBUGS, MLwiN (MCMC), R package MCMCglmm and SAS experimental procedure MCMC.Three data sets (the full data set and two sub-datasets) were analysed using basically two logistic random effects models with either one random effect for the center or two random effects for center and trial. For the ordinal outcome in the full data set also a proportional odds model with a random center effect was fitted.ResultsThe packages gave similar parameter estimates for both the fixed and random effects and for the binary (and ordinal) models for the main study and when based on a relatively large number of level-1 (patient level) data compared to the number of level-2 (hospital level) data. However, when based on relatively sparse data set, i.e. when the numbers of level-1 and level-2 data units were about the same, the frequentist and Bayesian approaches showed somewhat different results. The software implementations differ considerably in flexibility, computation time, and usability. There are also differences in the availability of additional tools for model evaluation, such as diagnostic plots. The experimental SAS (version 9.2) procedure MCMC appeared to be inefficient.ConclusionsOn relatively large data sets, the different software implementations of logistic random effects regression models produced similar results. Thus, for a large data set there seems to be no explicit preference (of course if there is no preference from a philosophical point of view) for either a frequentist or Bayesian approach (if based on vague priors). The choice for a particular implementation may largely depend on the desired flexibility, and the usability of the package. For small data sets the random effects variances are difficult to estimate. In the frequentist approaches the MLE of this variance was often estimated zero with a standard error that is either zero or could not be determined, while for Bayesian methods the estimates could depend on the chosen "non-informative" prior of the variance parameter. The starting value for the variance parameter may be also critical for the convergence of the M...

The contribution of the anaesthetist to risk‐adjusted mortality after cardiac surgery

“…The p values determining the significance of the fixed‐effect terms were calculated using the likelihood ratio test. Analyses were implemented using R (version 3.0.1, see http://www.r-project.org) 18, 19.…”

Relationship between Children’s Body Mass Index and Parents’ Obesity and Socioeconomic Status: A Multilevel Analysis Applied with Luxembourg Data

“…The first level concerns the individual characteristic of children, and the second level is represented by the socioeconomic characteristic and overweight's risk factors for parents (level-household). Assuming that the probability to be overweight or obese may be statistically dependent on both the variance of individual characteristics (level 1) and the parent's characteristics (level 2), we considered a multilevel logistic regression analysis based on a logit link function and a function of binomial distribution [35]- [38]. Statistical analyses were performed using SAS Software 9.2, and all results were weighted with an effect relative to the sample size.…”

Vol. 16, No. 2 (Full Issue)

“…Although the LME4, MCMCglmm, and PROC GLIMMIX packages were described for estimation in various binary GLMM models (Kim, Choi, & Emery, 2013;Li, Lingsma, Steyerberg, & Lesaffre, 2011;Zhang et al, 2011), the performance of the ORDINAL package has not yet been reported for binary outcomes nor for the calculation of agreement measures. Our focus in this paper is to explore the use of these four aforementioned packages in R and SAS to calculate the measures of agreement for multiple raters classifying test results using a binary scale.…”