Design and analysis of three‐arm parallel cluster randomized trials with small numbers of clusters

Abstract

In this article, we review and evaluate a number of methods used in the design and analysis of small three‐arm parallel cluster randomized trials. We conduct a simulation‐based study to evaluate restricted randomization methods including covariate‐constrained randomization and a novel method for matched‐group cluster randomization. We also evaluate the appropriate modelling of the data and small sample inferential methods for a variety of treatment effects relevant to three‐arm trials. Our results indicate that small‐sample corrections are required for high (0.05) but not low (0.001) values of the intraclass correlation coefficient and their performance can depend on trial design, number of clusters, and the nature of the hypothesis being tested. The Satterthwaite correction generally performed best at an ICC of 0.05 with a nominal type I error rate for single‐period trials, and in trials with repeated measures type I error rates were between 0.04 and 0.06. Restricted randomization methods produce little benefit in trials with repeated measures but in trials with single post‐intervention design can provide relatively large gains in power when compared to the most unbalanced possible allocations. Matched‐group randomization improves power but is not as effective as covariate‐constrained randomization. For model‐based analysis, adjusting for fewer covariates than were used in a restricted randomization process under any design can produce non‐nominal type I error rates and reductions in power. Where comparisons to two‐arm cluster trials are possible, the performance of the methods is qualitatively very similar.

Constrained randomization and statistical inference for multi‐arm parallel cluster randomized controlled trials

“…With more than two arms, Ciolino et al 19 proposed a class of P ‐value based balance metrics, and concluded that Kruskal‐Wallis test P ‐value with a threshold () leads to acceptable balance. Watson et al 8 extended the balance metric in Raab and Butcher 11 for multi‐arm cRCTs as the sum of the cluster‐level standardized mean differences across all arms, and followed Li et al 18 to choose the lowest of the randomization space (in terms of the balance score) for constrained randomization. In the current study, we wish to control the maximum degree of the between‐arm imbalance and therefore propose an alternative extension of the metric of Raab and Butcher 11 .…”

“…In addition, because there is little prior discussion on how to carry out randomization‐based inference in multi‐arm cRCTs, and given randomization‐based inference could be naturally coupled with constrained randomization, 17,18 we develop most‐powerful randomization tests in Section 3.2, extending the approach of Braun and Feng 26 to multiple arms and additional null hypotheses. Our evaluation assumes a cross‐sectional design with only a single post‐treatment outcome observation for each individual in each cluster, which resembles the TESTsmART study and represents the scenario where constrained randomization provides the maximum benefit for covariate balance 8 . We do not evaluate repeated cross‐sectional or cohort designs, but note that the horizontal before‐after comparisons in these alternative designs already offer some protection against between‐cluster imbalance.…”

“…That is, when a joint test for the global null is performed, the F ‐statistic should be referenced to a (central) F ‐distribution with DoF equal to (, # of cluster‐level covariates). Similarly, for the separate null of each pairwise comparison (), we use the t ‐test with the between‐within DoF # of cluster‐level covariates 8 . In two‐arm cRCTs conducted under constrained randomization, Li et al 18 demonstrated that the t ‐test with the between‐within DoF preserves the type I error rate, and similar findings (albeit in the absence of multiplicity adjustment) were observed in Watson et al 8 for three‐arm cRCTs conducted under constrained randomization.…”

“…While the choice between the and balance metrics does not lead to substantial differences for statistical inference, they found that a smaller randomization space (such as 10% of the simple randomization space with the smallest balance score) could improve the power of the model‐based and randomization‐based tests. Ciolino et al 19 and Watson et al 8 extended the balance metrics to multi‐arm cRCTs. However, the consideration of alternative balance metrics as well as the effect of constrained subspace sizes on the validity of analytical methods were not fully articulated for multi‐arm cRCTs in these prior studies (see Web Table 1 for comparison).…”

Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review

“…Multiple studies found that increasing cluster size variability led to decreasing power [32][33][34][35], and that the magnitude of the power loss depended on the value of the ICC [33,36]. Eldridge et al [17] concluded that if the cluster size CV is less than 0.23, there is no need to account for the effects of variable cluster size in the sample size calculation.…”