How Rigorous is Active Learning Research in STEM Education? An Examination of Key Internal Validity Controls in Intervention Studies

Abstract

Active learning is a popular approach to teaching and learning that has gained traction through research on STEM educational improvement. There have been numerous university- and national/international-level efforts focused on transitioning courses from the lecture method to active learning. However, despite these large-scale changes, the active learning literature has not been assessed on its methodological rigor to ensure instructional recommendations are rooted in rigorous research studies. The purpose of the present review was to determine areas of strengths and areas in need of improvement and to provide specific recommendations on how to continue or improve active learning research to strengthen the respective literature base and increase confidence in results. We assessed the articles included in the Freeman et al. (PNAS, 111:8410–8415, 2014) meta-analysis as well as a random sample of more recent active learning articles (2015–2022) on 12 internal validity controls (i.e., control procedure used to prevent a threat to the internal validity of a study). Results indicated that there were high percentages of articles that did not meet each internal validity control. In fact, no articles from the Freeman et al. meta-analysis and no sampled 2015–2022 articles met each of the 12 internal validity controls. Therefore, the active learning literature contains numerous internal validity control issues that need to be addressed if we are to determine the extent to which active learning interventions are effective and if there are any boundary conditions for when particular active learning interventions are or are not effective.

Self-Reported Limitations in Chemistry Education Research: Providing Specific and Contextualized Limitations Supports Researchers and Practitioners

“…Threats to a study’s validity may derive from confounding variables, participant attrition, biases attributed to participants’ participation (i.e., the Hawthorne effect), or other factors. These possible study limitations can impact the learning sciences, as demonstrated by a recent report detailing numerous issues regarding internal validity for the body of work supporting active learning interventions . Importantly, limitations are often differentiated from delimitations, which are intentional boundaries set for a study (e.g., focusing a study on a specific population, selecting a specific context, narrowing the parameters or variables considered, etc.)…”

“…These possible study limitations can impact the learning sciences, as demonstrated by a recent report detailing numerous issues regarding internal validity for the body of work supporting active learning interventions. 14 Importantly, limitations are often differentiated from delimitations, which are intentional boundaries set for a study (e.g., focusing a study on a specific population, selecting a specific context, narrowing the parameters or variables considered, etc.). 12 As an implication of this, narrowing the scope of a study often introduces limitations and concerns related to external validity and generalizability.…”

Distributed Active Learning (DAL): An Approach to Reduce Class Size in Large Enrollment Courses

“…11 However, evaluating the effectiveness of active learning as a pedagogical approach is not a simple matter. 12,13 One complicating factor is that classroom instruction is one component in the larger course structure that includes both pre-and postclass elements. Higher structure courses have more robust pre-and postclass elements, such as preclass reading assignments that are graded.…”

“…Findings from the investigations of Freeman and Theobald are compelling, have increased enthusiasm for active learning, and have mitigated earlier concerns that active learning has “uneven support” . However, evaluating the effectiveness of active learning as a pedagogical approach is not a simple matter. , One complicating factor is that classroom instruction is one component in the larger course structure that includes both pre- and postclass elements. Higher structure courses have more robust pre- and postclass elements, such as preclass reading assignments that are graded. ,,, If a class includes both active learning and a high course structure it is difficult to separate these factors.…”