Promoting adolescents’ comprehension of text: A randomized control trial of its effectiveness.

Abstract

Promoting Adolescents' Comprehension of Text (PACT) is a text-and discourse-based set of instructional practices that engage students with disciplinary texts as a means of building content knowledge and improving reading comprehension. PACT's efficacy has been the subject of extensive previous trials. The purpose of this study was to evaluate its effectiveness in a school-randomized design using stratified balanced sampling to assemble a representative sample of schools from the population of middle schools that teach U.S. history. The population-level effect estimates on a posttest of knowledge acquisition were 0.45 using weighted mixed effects estimation and 0.37 using weighted ordinary least squares. At follow-up, the effects were 0.53 based on weighted mixed effects estimation and 0.33 based on robust variance estimation. Furthermore, although treatment's effects on content area reading comprehension and broad reading comprehension were not statistically significant, the sample-based treatment effects (g = 0.15 for content area reading and 0.14 for broad reading) were not trivial when evaluated in the context of other studies of literacy instruction with older readers. Our findings represent the most reliable estimates of PACT's average studentlevel effects when the program is implemented at the school level. In addition, these estimates are for the program when implemented in "real-world" settings, versus in the more controlled conditions typical of efficacy designs. Finally, the results further replicate the PACT's instructional practices, in this sample and in the population of middle schools that teach U.S. history. Educational Impact and Implications StatementPromoting Adolescents' Comprehension of Text (PACT) is a text-and discourse-based set of instructional practices that engage students with disciplinary texts as a means of building content knowledge and improving reading comprehension. Earlier studies support PACT's impact when teachers and researchers work together closely. Findings of the present study indicate that PACT is also effective when the instructional practices are implemented in "real-world" settings, where researchers are not directly supporting classroom teachers. Our results also suggest that PACT is likely to have an effect, on average, in the population of U.S. public middle schools that teach U.S. history.

Promoting Adolescents’ Comprehension of Text: Efficacy and Effectiveness

“…Study 4, effectiveness RCT: Middle school social studies/ history. Study 4 examined the effectiveness of PACT and involved multiple cohorts of schools collected over several years with schools assigned randomly to the PACT or BaU condition (Roberts et al, 2023). The sample design, recruitment, and generalization of this effectiveness study are described in the Roberts et al paper.…”