Does the group in group psychotherapy matter? A meta-analysis of the intraclass correlation coefficient in group treatment research.

Abstract

Objective: Over the last 3 decades, group treatment researchers have become increasingly knowledgeable of the impact of within-group dependency on analyses of group treatment data and of mutual influence processes that occur within therapy groups. Despite these advancements, there remains a lack of consensus on the magnitude of mutual influence, or group effects, in group treatment research. As such, this study sought to estimate the size of group effects on members' posttreatment outcomes by meta-analyzing the intraclass correlation coefficients (ICCs) in group treatment research. In addition, we tested several moderators of the ICC, including outcome type, outcome reactivity, outcome specificity, group format, treatment length, and group size. Method: Using robust variance estimations, we metaanalyzed 169 effect sizes from 37 group treatment studies. Results: Findings indicated an average ICC of 0.06. Group size, group format, treatment length, outcome specificity, and outcome type did not significantly moderate the ICC; however, we did find evidence to suggest that the ICC varies as a function of outcome reactivity, with observer-rated outcome measures resulting in the largest ICC. Conclusion: These findings suggest that interdependence in group treatment research is an important concept both theoretically and statistically. What is the public health significance of this article?This study found that therapy groups account for 6% of the variability in group treatment outcomes. Outcome reactivity significantly moderated the intraclass correlation coefficient (ICC) in group treatment research, such that the largest ICCs were associated with observer-rated outcome measures. Interdependence in group treatment research is important both theoretically and statistically.

The Potential Influence of Group Membership on Outcomes in Indicated Cognitive-Behavioral Adolescent Depression Prevention

“…Our results showed that group effects explained a significant proportion of posttest depressive symptom levels (16%) even after controlling for baseline symptoms. This group-level variation is higher than variation reported in previous adult treatment studies (6-7%) [5,6,19]. In other words, some groups appeared more successful than others at reducing existing depressive symptoms during receipt of the group intervention.…”

“…This group-level phenomenon is known as a "clustering effect" and can be estimated using the intra-class correlation (ICC), which represents the ratio of group-level variation to total variation in the outcome [4]. Previous studies in adults have demonstrated the existence of significant clustering effects (or ICC) in group-based interventions (e.g., 6-7% of variance [5,6]).…”

Do We Overestimate the Within-Variability? The Impact of Measurement Error on Intraclass Coefficient Estimation

“…Likewise, Kivlighan et al (2019) examined 169 effect sizes from 37 group treatment studies, stressed group therapy members mutually influence their group posttreatment outcomes (e.g., depression, etc. ), and found that group membership explains about 6% of variance [ICC(1) = 0.06; Cohen's d of about 0.47].…”

“…The intraclass coefficient (1) 1 (ICC(1)) reflects the degree of resemblance of micro-level units (e.g., events) within macro-level units (e.g., individuals), and is calculated as between-variance (e.g., variance, existing between groups or individuals) divided by total-variance (e.g., the sum of between-variance and the variance existing within groups or individuals; e.g., Fisher, 1934;Shrout and Fleiss, 1979;Snijders and Bosker, 2012). It is often used to provide insights of the magnitude of variance on different levels to test and inform psychological theories (e.g., Bliese et al, 2002;Castro, 2002;Nook et al, 2018;Kivlighan et al, 2019;Podsakoff et al, 2019). For example, Podsakoff et al (2019) showed that many psychological constructs (e.g., personality, coping, or job characteristics) vary substantially within-person (at least 40%), even though they "have historically been treated as between-person phenomena" (Podsakoff et al, 2019, p. 737).…”

“…For example, Podsakoff et al (2019) showed that many psychological constructs (e.g., personality, coping, or job characteristics) vary substantially within-person (at least 40%), even though they "have historically been treated as between-person phenomena" (Podsakoff et al, 2019, p. 737). Hence, Kivlighan et al (2019) showed that group therapy members mutually influence their group posttreatment outcomes (e.g., depression, etc. ), and found that group membership explains about 6% of variance [ICC(1) = 0.06; Cohen's d of about 0.47].…”