Clustered data with small sample sizes: Comparing the performance of model-based and design-based approaches

McNeish, Daniel; Harring, Jeffery R.

doi:10.1080/03610918.2014.983648

Cited by 72 publications

(56 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As has been demonstrated in previous research (e.g., Browne & Draper, 2006;McNeish & Harring, 2015), ML and REML tended to have coverage intervals that are shorter than the nominal rate, especially for predictors involving a variable at Level 2. Use of a Kenward-Roger correction was largely able to address this limitation and provided coverage rates within the nominal  Prior to the DEFT correction, the coverage rate for the treatment effect with  to  observations per cluster was % and with  to  observations per cluster it was % across all number-of-cluster conditions.…”

Section: Multilevel Modelssupporting

confidence: 76%

“…Mancl-DeRouen had coverage rates that were much closer to nominal rates but were still a little too short, particularly for predictors at Level 1 with fewer than 10 clusters. Morel-Bokossa-Neerchal performed the best of all the GEE methods although the coverage rates tended to consistently be on the high end of Bradley's range, as was similarly found in McNeish and Harring (2015).…”

Section: Generalized Estimating Equationssupporting

confidence: 60%

See 1 more Smart Citation

Modeling Clustered Data with Very Few Clusters

McNeish

Stapleton

2016

Multivariate Behavioral Research

324

279

View full text Add to dashboard Cite

Small-sample inference with clustered data has received increased attention recently in the methodological literature, with several simulation studies being presented on the small-sample behavior of many methods. However, nearly all previous studies focus on a single class of methods (e.g., only multilevel models, only corrections to sandwich estimators), and the differential performance of various methods that can be implemented to accommodate clustered data with very few clusters is largely unknown, potentially due to the rigid disciplinary preferences. Furthermore, a majority of these studies focus on scenarios with 15 or more clusters and feature unrealistically simple data-generation models with very few predictors. This article, motivated by an applied educational psychology cluster randomized trial, presents a simulation study that simultaneously addresses the extreme small sample and differential performance (estimation bias, Type I error rates, and relative power) of 12 methods to account for clustered data with a model that features a more realistic number of predictors. The motivating data are then modeled with each method, and results are compared. Results show that generalized estimating equations perform poorly; the choice of Bayesian prior distributions affects performance; and fixed effect models perform quite well. Limitations and implications for applications are also discussed.

show abstract

Section: Multilevel Modelssupporting

confidence: 76%

Section: Generalized Estimating Equationssupporting

confidence: 60%

Modeling Clustered Data with Very Few Clusters

McNeish

Stapleton

2016

Multivariate Behavioral Research

324

279

View full text Add to dashboard Cite

show abstract

“…When the effects were significant (α = 0.05), Tukey's Honest Significant Difference test was performed. Additionally, Kenward-Roger correction was applied for reducing small sample bias [26]. Statistical analyses were conducted using the packages "plyr", "ggplot2", "lme4" and "lmerTest" for the linear models; and the packages: "rmisc", "rcmdmisc", "plyr", "ggplot2", "car", "multcompView" and "lsmeans" for the linear mixed models implemented in the R software version 3.3.3 [27].…”

Section: Statistical Analysesmentioning

confidence: 99%

Physicochemical and textural quality attributes of gluten-free bread formulated with guar gum

Encina‐Zelada

Cadavez

Monteiro

et al. 2018

Eur Food Res Technol

View full text Add to dashboard Cite

The objective of this study was to assess the combined effect of guar gum (GG) and water content (WC) on the rheological properties of batter, and the physicochemical and textural properties of bread. Batches of gluten-free bread used a base formulation of rice (50%), maize (30%) and quinoa flour (20%), with different levels of GG (2.5, 3.0 or 3.5%) and water (90, 100 or 110%) in a full factorial design. Higher GG doses (p < 0.001) tended to produce batters of lower stickiness, work of adhesion and cohesive strength; yet, of higher firmness, consistency, cohesiveness and viscosity index. These batters yielded loaves of lower (p < 0.001) specific volume and baking loss; and crumbs of lower (p < 0.001) a w , pH, mean cell area, void fraction, mean cell aspect ratio; and higher (p < 0.001) hardness, adhesiveness, springiness, cohesiveness, chewiness, resilience, mean cell density, cell size uniformity and mean cell compactness. The sticker and less consistent batters produced with higher WC rendered larger bread loaves of softer and more cohesive and springy/resilient crumbs with greater mean cell size and void fraction. Gluten-free loaves of good appearance in terms of higher specific volume, lower crumb hardness, higher crumb springiness, and open grain visual texture were obtained in formulations with 110% WC and GG doses between 2.5 and 3.0%.

show abstract

“…Several variations of CRSEs are available, at least for a single-level model (McNeish, 2014;McNeish & Harring, 2017;Raudenbush & Bryk, 2002). Mplus provides a CRSE procedure referred to as type = complex, which entails fitting a single-level model to data and correcting the standard errors for clustering at a higher level.…”

Section: Three Approaches To Analyze Ms-crt Datamentioning

confidence: 99%

“…A relatively recent approach, cluster bootstrapping, has been shown to produce results similar to those of multilevel modeling (MLM), but the results are computationally demanding, especially within a Monte Carlo simulation study (Huang, 2016). In brief, MLM, as a completely model-based approach, and generalized estimating equations (GEE) and clusterrobust standard errors (CRSE), as design-based approaches, are possible alternatives for analyzing MS-CRT data (McNeish, 2014;McNeish & Harring, 2017;McNeish & Wentzel, 2017). Among these three methods, MLM is currently the predominant method in the field of the social sciences (Bauer & Sterba, 2011;McNeish, Stapleton, & Silverman, 2017).…”

mentioning

confidence: 99%

Comparison of model- and design-based approaches to detect the treatment effect and covariate by treatment interactions in three-level models for multisite cluster-randomized trials

2018

View full text Add to dashboard Cite

In this study, we evaluated the estimation of three important parameters for data collected in a multisite cluster-randomized trial (MS-CRT): the treatment effect, and the treatment by covariate interactions at Levels 1 and 2. The Level 1 and Level 2 interaction parameters are the coefficients for the products of the treatment indicator, with the covariate centered on its Level 2 expected value and with the Level 2 expected value centered on its Level 3 expected value, respectively. A comparison of a model-based approach to design-based approaches was performed using simulation studies. The results showed that both approaches produced similar treatment effect estimates and interaction estimates at Level 1, as well as similar Type I error rates and statistical power. However, the estimate of the Level 2 interaction coefficient for the product of the treatment indicator and an arithmetic mean of the Level 1 covariate was severely biased in most conditions. Therefore, applied researchers should be cautious when using arithmetic means to form a treatment by covariate interaction at Level 2 in MS-CRT data.

show abstract

Clustered data with small sample sizes: Comparing the performance of model-based and design-based approaches

Cited by 72 publications

References 28 publications

Modeling Clustered Data with Very Few Clusters

Modeling Clustered Data with Very Few Clusters

Physicochemical and textural quality attributes of gluten-free bread formulated with guar gum

Comparison of model- and design-based approaches to detect the treatment effect and covariate by treatment interactions in three-level models for multisite cluster-randomized trials

Contact Info

Product

Resources

About