From problem-solving to sensemaking: A comparative meta-analysis of preparatory approaches for future learning

Abstract

Against the backdrop of a growing body of research showing the effectiveness of problem-solving activities followed by instruction (PS-I), we report a meta-analysis of the effectiveness of three broad categories of preparatory activities on future learning from instruction: (a) problem-solving followed by instruction (PS-I), (b) scaffolded problem-solving followed by instruction (+PS-I), or (c) an alternative sensemaking activity followed by instruction (!PS-I)? We examined 118 experimental comparisons spanning 33 articles that compared PS-I with +PS-I and !PS-I designs. Although scaffolding was descriptively associated with a small effect size, there was no significant difference relative to PS-I (Hedge’s g -0.08 [95% CI -0.20, 0.04]). Additionally, PS-I exhibited a non-significant moderate effect (Hedge’s g 0.22 [95% CI -0.06, 0.51]) compared to !PS-I. Bayesian analyses strongly favored the null hypothesis for the comparison of PS-I with +PS-I (suggesting a 99% probability of the difference in effect between these designs being less than 0.2), while it suggested a 40.37% probability of at least a moderate effect favoring PS-I relative to !PS-I. Further, the estimation of true effect sizes after accounting for the publication bias suggested moderate effect sizes in favor of PS-I, when considering both comparison conditions +PS-I (Hedge’s g 0.55) and !PS-I (Hedge’s g 0.64).

Enriching problem-solving followed by instruction with explanatory accounts of emotions

“…In summary, there exists a research gap in investigating the interplay between cognition and fine-grained affect for instructional approaches following a problem-solving first pedagogy, especially via the advances in the reported affective computing methods. Given the growing effectiveness of PS-I approaches (Sinha & Kapur, 2021b, 2021c, it is important to understand and triangulate evidence for underlying affective and cognitive mechanisms that drive learning benefits. This article's focus on a finegrained investigation into emotional states is one way of approaching this important theoretical question.…”

“…Since the seminal work on preparation for future learning (Schwartz & Bransford, 1998), educational research has taken great strides in testing approaches that afford students sensemaking opportunities before formal instruction (see Sinha and Kapur (2021b) and Sinha and Kapur (2021c) for meta-analyses of the field). Preparatory problem-solving is one such sensemaking activity that is posited to activate students' relevant prior knowledge, increase awareness of knowledge gaps, and their desire to know the canonical answer (Loibl et al, 2017).…”

Comparing the effectiveness of preparatory activities that help undergraduate students learn from instruction

“…We compare productive failure to three alternative means of preparing students for instruction. Two of the three alternative conditions-contrasting cases and worked examples-appear more frequently in literature (Sinha & Kapur, 2021b). Therefore, the effectiveness of these two approaches, at least in comparison to I-PS or other proxies of "typical" instruction, is relatively established.…”

“…However, the differential impact of PS-I in comparison to instructional designs involving alternative forms of preparatory activity, particularly for learners who are not novices, is unclear. To emphasize this point, the effectiveness of preparatory activities on learning is largely unexplored in the context of university learning: a meta-analysis of the field (Sinha & Kapur, 2021b) found huge variance in effect sizes between the handful of studies that have evaluated the efficacy of PS-I for university students, predominantly in comparison to worked examples followed by instruction (e.g., Cook, 2017;Newman & DeCaro, 2019;Roelle & Berthold, 2015). Thus, our aim was to investigate the impact of PS-I in comparison to alternative preparatory designs, particularly in the university context with non-novice learners.…”

“…Some of these alternative designs may well be more effective, or more effective for particular learners, such as students who are not novices. In a recent meta-analysis of PS-I studies (Sinha & Kapur, 2021b), we examined 118 experimental comparisons of PS-I with alternative instructional designs that afford preparation for follow-up instruction. Out of those, we found 58 comparisons of PS-I with a different preparatory activity prior to instruction-for example, preparing students for instruction by reading or summarizing text, studying worked examples, problem-posing, ranking contrasting cases, and so on.…”