Using Complier Average Causal Effect Estimation to Determine the Impacts of the Good Behavior Game Preventive Intervention on Teacher Implementers

Abstract

Complier average causal effect (CACE) analysis is a causal inference approach that accounts for levels of teacher implementation compliance. In the current study, CACE was used to examine one-year impacts of PAX good behavior game (PAX GBG) and promoting alternative thinking strategies (PATHS) on teacher efficacy and burnout. Teachers in 27 elementary schools were randomized to PAX GBG, an integration of PAX GBG and PATHS, or a control condition. There were positive overall effects on teachers' efficacy beliefs, but high implementing teachers also reported increases in burnout across the school year. The CACE approach may offer new information not captured using a traditional intent-to-treat approach.

Game On—Complier Average Causal Effect Estimation Reveals Sleeper Effects on Academic Attainment in a Randomized Trial of the Good Behavior Game

“…However, no significant intervention effect among high compliers was found. While this is inconsistent with existing research (Berg et al 2017;O'Connell et al 2009), it may well be due to the smaller sample size of high compliers (n = 333, 21.3%), and/or the computational demand arising from the use of FIML within multilevel mixture modeling (Panayiotou et al 2019). To test this, we ran post hoc single-level models accounting for clustering through Type = Complex, although results were unchanged.…”

“…However, meeting the exclusion restriction assumption was less likely given the arbitrary thresholds used to define compliance. For instance, students could still potentially be affected by the GBG even at lower levels of exposure (Berg et al 2017). Although relaxing this assumption is possible with the inclusion of strong predictors of compliance (Jo 2002a), the effectiveness of this method has been less studied within multilevel CACE (Jo et al 2008), and has received no empirical support within multilevel CACE with missing data (Jo et al 2010).…”

“…Thus, it is currently unknown whether variability in GBG implementation influences its impact (Berg et al 2017), and in particular, the moderating role of dosage is yet to be examined. As this variability is inevitable in school-based interventions (Durlak 2016), it is likely that traditional ITT analyses provide a biased estimate of their effects (Jo and Muthén 2001;.…”

“…As this variability is inevitable in school-based interventions (Durlak 2016), it is likely that traditional ITT analyses provide a biased estimate of their effects (Jo and Muthén 2001;. As a supplement to ITT models, complier average causal effect (CACE) estimation offers a robust, unbiased means through which to examine intervention effects while taking levels of implementation into account (Berg et al 2017). However, it has been given "little to no attention in school psychology" (Peugh and Toland 2017, p. 5).…”

“…Compliance Following the framework suggested by Angrist et al (1996), a binary indicator of compliance (0 = non-compliers; 1 = compliers) is required in the intervention group for the identification of the latent compliance variable. In the absence of a verified cutoff, sensitivity analyses were conducted, where compliance was defined in one of two ways (as in Berg et al 2017):…”