The Design of Cluster Randomized Trials With Random Cross-Classifications

Moerbeek, Mirjam; Safarkhani, Maryam

doi:10.3102/1076998617730303

Cited by 8 publications

(13 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Despite the increasing amount of cluster-randomized trials in education and other social sciences in recent years, there have been relatively few studies that actively modeled crossed random effects, meaning that results of many treatment effects in the literature may need to be adjusted. For example, Moerbeek and Safarkhani (2018) provided an example where soldiers are randomly assigned to different therapeutic approaches with different therapists, but the soldiers are also naturally nested within army units. Another common example in education is when classrooms of students are randomly assigned, but students are cross-classified by current classroom (e.g., second grade) and also past-year classroom (e.g., first grade), yet primary studies may not have accounted for the shared variance of Grade 1 classroom/teacher effects.…”

Section: Discussionmentioning

confidence: 99%

Correcting Fixed Effect Standard Errors When a Crossed Random Effect Was Ignored for Balanced and Unbalanced Designs

Lai

2019

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

Previous studies have detailed the consequence of ignoring a level of clustering in multilevel models with straightly hierarchical structures and have proposed methods to adjust for the fixed effect standard errors ( SEs). However, in behavioral and social science research, there are usually two or more crossed clustering levels, such as when students are cross-classified by schools and neighborhoods, yet it is not uncommon that researchers focus only on one level of clustering. Using the generalized least squares framework, in this study, we derive the bias in the fixed effect SE estimators when one crossed random effect is omitted. We then showed, using data from the Scotland Neighborhood Study, how one can correct for the SEs and obtain corrected statistical inference when a misspecified two-level model was used in a primary study, which is useful when evaluating observational studies or cluster randomized trials that ignored a crossed random effects or when conducting meta-analyses. In addition, our analytic results provide theoretical insights on how one can quantify imbalance with cross-classified data by the strength of association between the two-crossed random effects in a contingency table and how the degree of imbalance relates to the correction factor for the fixed effect SEs.

show abstract

Section: Discussionmentioning

confidence: 99%

Correcting Fixed Effect Standard Errors When a Crossed Random Effect Was Ignored for Balanced and Unbalanced Designs

Lai

2019

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

show abstract

“…where = − − 2 for the two-level CRT, , /2 (0) is the statistic associated with central distribution with degrees of freedom and probability /2, ( ) is the statistic associated with non-central distribution with non-centrality parameter = * / ( * ), degrees of freedom , and and are Type I and Type II error rates (see, Hedges & Rhoads, 2010;Moerbeek & Safarkhani, 2018). In what follows we will demonstrate how to estimate variance parameters and how to calculate parameters needed in ( * ) formula.…”

Section: Standard Error Formula Under Balanced Sample Size and Homogementioning

confidence: 99%

“…Tip hata oranlarıdır (bkz. Hedges & Rhoads, 2010;Moerbeek & Safarkhani, 2018). Aşağıda, varyans parametrelerinin nasıl kestirileceği ve ( * ) formülünde ihtiyaç duyulan parametrelerin nasıl hesaplanacağı gösterilmiştir.…”

Section: Dengeli öRneklem Büyüklüğü Ve Homojen Varyans Kapsamında Standart Hata Formülüunclassified

Estimation and Standardization of Variance Parameters for Planning Cluster-Randomized Trials: A Short Guide for Researchers

Buluş

Şahın

2019

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

A review of literature covering the past decade indicates a shortage of cluster-randomized trials (CRTs) in education and psychology in Turkey, the gold standard that is capable of producing high-quality evidence for high-stake decision making when individual randomization is not feasible. Scarcity of CRTs is not only detrimental to collective knowledge on the effectiveness of interventions but also hinders efficient design of such studies as prior information is at best incomplete or unavailable. In this illustration, we demonstrate how to estimate variance parameters from existing data and transform them into standardized forms so that they can be used in planning sufficiently powered CRTs. The illustration uses publicly available software and guides researchers step by step via introducing statistical models, defining parameters, relating them to notations in statistical models and power formulas, and estimating variance parameters. Finally, we provide example statistical power and minimum required sample size calculations.

show abstract

“…) for a two-tailed hypothesis test were computed as suggested in the existing literature (see, Bloom, 2006, p. 4; see also Dong & Maynard, 2013;Hedges & Rhoads, 2010;Moerbeek & Safarkhani, 2018).…”

Section: Discussionmentioning

confidence: 99%

Statistical power and precision of experimental studies originated in the Republic of Turkey from 2010 to 2020: Current practices and some recommendations

Buluş

Koyuncu

2021

Participatory Educational Research

View full text Add to dashboard Cite

This study systematically reviews randomly selected 155 experimental studies in education field originated in the Republic of Turkey between 2010 and 2020. Indiscriminate choice of sample size in recent publications prompted us to evaluate their statistical power and precision. First, above and beyond our review, we could not identify any large-scale experiments such as cluster-randomized or multisite randomized trials, which overcome shortcomings of small-scale experiments, better suit to the organizational structure of the education field, nevertheless require far greater effort and financial resources. Second, none of the small-scale experiments has reported or conducted ex-ante power analysis. Third, results indicate that studies are sufficiently powered to detect medium effects and above (Cohen's d ≥ 0.50), however they are underpowered to detect small effects (Cohen's d ≤ 0.20). Trends in the past ten years indicate precision remained unchanged. We made several recommendations to increase the precision of experimental designs and improve their evidential values: Determine sample size prior to an experiment with power analysis routine, randomize subjects / clusters to obtain unbiased estimates, collect pre-test information and other relevant covariates, adjust for baseline differences beyond covariate control, document attrition, report standardized treatment effect and standardized variance parameters. Findings should be interpreted considering minimum effects in education that are relevant to education policy and practice.

show abstract

The Design of Cluster Randomized Trials With Random Cross-Classifications

Cited by 8 publications

References 54 publications

Correcting Fixed Effect Standard Errors When a Crossed Random Effect Was Ignored for Balanced and Unbalanced Designs

Correcting Fixed Effect Standard Errors When a Crossed Random Effect Was Ignored for Balanced and Unbalanced Designs

Estimation and Standardization of Variance Parameters for Planning Cluster-Randomized Trials: A Short Guide for Researchers

Statistical power and precision of experimental studies originated in the Republic of Turkey from 2010 to 2020: Current practices and some recommendations

Contact Info

Product

Resources

About