Unpacking Complex Mediation Mechanisms and Their Heterogeneity Between Sites in a Job Corps Evaluation

Abstract

This study aims to test the theory underlying Job Corps, one of the largest education and training programs in the U.S. serving disadvantaged youth. Central to the program are vocational training and general education that serve as two concurrent mediators transmitting the program impact on earnings. To distinguish the relative contribution of each, we develop methods for decomposing the Job Corps impact on earnings into an indirect effect transmitted through vocational training, an indirect effect transmitted through general education, and a direct effect attributable to supplementary services. We further ask whether general education and vocational training reinforce each other and produce a joint impact greater than the sum of the two separate pathways. Moreover, we examine the heterogeneity of each causal effect across all the Job Corps centers. This article presents concepts and methods for defining, identifying, and estimating not only the population averages but also the between‐site variance of these causal effects. Our analytic procedure incorporates a series of weighting strategies to enhance the internal and external validity of the results and assesses the sensitivity to potential violations of the identification assumptions.

Posttreatment Confounding in Causal Mediation Studies: A Cutting-Edge Problem and a Novel Solution via Sensitivity Analysis

“…Finally, some posttreatment confounders precede the focal mediator M, whereas some others are concurrent to M. This paper is restricted to the former case. Qin et al (2021) proposed a sensitivity analysis strategy for causal mediation studies that involve two concurrent mediators that are not conditionally independent. Readers may also consider alternative strategies proposed by Imai and Yamamoto (2013), Daniel et al (2015), and Albert et al (2019).…”

“…Qin et al. (2021) proposed a sensitivity analysis strategy for causal mediation studies that involve two concurrent mediators that are not conditionally independent. Readers may also consider alternative strategies proposed by Imai and Yamamoto (2013), Daniel et al.…”

Estimating Causal Mediation Effects in Multiple-Mediator Analyses with Clustered Data

“…Future research is warranted on sensitivity analysis for the interventional (in)direct effects with clustered data, particularly the sensitivity to unmeasured individual-level confounders. It could be an important future direction to further extend the literature on sensitivity analysis for causal mediation effects to the multiple-mediator analysis with clustered data (e.g., Liu et al, 2023;Park & Esterling, 2021;Qin et al, 2021;Rubinstein, Branson, & Kennedy, 2023;Tofighi et al, 2019) In conclusion, we hope that this study can provide insights and add to researchers' toolbox for estimating the interventional indirect and direct effects through multiple mediators with clustered data.…”

“…Note that in causal inference with clustered data, the plausibility of the SUTVA requires careful consideration (e.g., Hong & Raudenbush, 2006;Qin et al, 2021;VanderWeele et al, 2013). Particularly, the no-interference component of the SUTVA assumes that the potential outcomes and potential mediators of an individual are unaffected by other individuals, even other individuals in the same cluster.…”

“…In previous literature, similar weighting-based estimators of causal mediation effects have been available for single-level data (e.g., Hong, 2010;Hong et al, 2015;Loh et al, 2020;Nguyen et al, 2023;Tchetgen Tchetgen & Shpitser, 2012;Xia & Chan, 2022); for clustered data, Qin et al (2019Qin et al ( , 2021 developed weighting-based estimators for the natural (in)direct effects with a single mediator and with multiple mediators that are conditionally independent (given treatment and covariates). To our knowledge, for estimating the interventional (in)direct effects (Eqs.…”