A comparison of statistical methods for the analysis of binary repeated measures data with additional hierarchical structure

Abstract

The objective of the study was to compare statistical methods for the analysis of binary repeated measures data with an additional hierarchical level. Random effects true models with autocorrelated ($\rho=1$, 0.9 or 0.5) subject random effects were used in this simulation study. The settings of the simulation were chosen to reflect a real veterinary somatic cell count dataset, except that the within--subject time series were balanced, complete and of fixed length (4 or 8 time points). Four fixed effects parameters were studied: binary predictors at the subject and cluster levels, respectively, a linear time effect, and the intercept. The following marginal and random effects statistical procedures were considered: ordinary logistic regression (OLR), alternating logistic regression (ALR), generalized estimating equations (GEE), marginal quasi-likelihood (MQL), penalized quasi-likelihood (PQL), pseudo likelihood (REPL), maximum likelihood (ML) estimation and Bayesian Markov chain Monte Carlo (MCMC). The performance of these estimation procedures was compared specifically for the four fixed parameters as well as variance and correlation parameters. The findings of this study indicate that in data generated by random intercept models ($\rho=1$), the ML and MCMC procedures performed well and had fairly similar estimation errors. The PQL regression estimates were attenuated while the variance estimates were less accurate than ML and MCMC, but the direction of the bias depended on whether binomial or extra-binomial dispersion was assumed. In datasets with autocorrelation ($\rho<1$), random effects estimates procedures gave downwards biased estimates, while marginal estimates were little affected by the presence of autocorrelation. The results also indicate that in addition to ALR, a GEE procedure that accounts for clustering at the highest hierarchical level is sufficient.

A simulation study to assess the impact missing values on the performance of different statistical methods for analysis of binary repeated measures data with an additional hierarchical structure

“…Random effects and marginal estimation procedures were selected based on their performance in the full and balanced simulated datasets (Masaoud and Stryhn, 2020). Random effects estimation procedures included several approximation algorithms, aimed at producing estimates close to the global ML estimate without actually computing the likelihood function (Breslow, 2003).…”

“…Marginal estimation procedures included GEE, generalized estimating equations, and some of its variants; for more details, see (Masaoud and Stryhn, 2020). For missing data scenarios involving drop-outs by an MAR process, a weighted generalized estimating equation (WGEE) procedure was employed to account for the bias induced by the MAR mechanism.…”

“…A GEE procedure may allow an MAR process to be ignored if the working correlation structure is specified correctly (Liang and Zeger, 1986;Jansen et al, 2006); see however (Preisser et al, 2002) for examples where this does not hold. The GEE procedure was set up with either an independence or exchangeable working correlation structure at the cluster (herd) level; results from (Masaoud and Stryhn, 2020) showed that GEE with these correlations at the cluster level performed well for balanced repeated measures data with an additional hierarchical structure. The calculations involved in the weighting scheme have been detailed elsewhere (Jansen et al, 2006;Molenberghs and Verbeke, 2005, Chapter 27).…”

“…Thus, the hierarchical structure is the clustering of cows in herds, the repeated measures are the monthly test records based on the milk samples, and the missing values are the incomplete records on each cow. A previous study by (Masaoud and Stryhn, 2020) targeted the added complexity of the additional hierarchical structure in a balanced full datasets setting, whereas the present study is focused on the missing values part. Generally, missingness in longitudinal data presents a potential source of bias.…”

“…In order to realistically reflect the choice an applied researcher faces when it comes to data analysis, only estimation procedures implemented in broadly accessible statistical software were considered for the study. Specifically, the following procedures previously studied for hierarchically structured binary repeated measures data (Masaoud and Stryhn, 2020) were included: maximum likelihood via numerical integration (ML), Bayesian Markov chain Monte Carlo (MCMC), penalized quasi-likelihood with binomial dispersion (PQL) and an extra-binomial dispersion (PQLx), ordinary logistic regression (OLR), alternating logistic regression (ALR), and weighted generalized estimating equations (WGEE). The adapted ALR macro for 3-level of clustering (Kunthel et al, 2014) is recently available when estimation of the association structure is of primary interest, though was not included in the present simulation study.…”