Assessing the Association Between Precourse Metrics of Student Preparation and Student Performance in Introductory Statistics: Results from Early Data on Simulation-Based Inference vs. Nonsimulation-Based Inference

Abstract

The recent simulation-based inference (SBI) movement in algebra-based introductory statistics courses (Stat 101) has provided preliminary evidence of improved student conceptual understanding and retention. However, little is known about whether these positive effects are preferentially distributed across types of students entering the course. We consider how two metrics of Stat 101 student preparation (pre-course performance on concept inventory and math ACT score) may or may not be associated with end of course student performance on conceptual inventories. Students across all preparation levels tended to show improvement in Stat 101, but more improvement was observed across all student preparation levels in early versions of a SBI course. Furthermore, students' gains tended to be similar regardless of whether students entered the course with more preparation or less. Recent data on a sample of students using a current version of an SBI course showed similar results, though direct comparison with non-SBI students was not possible. Overall, our analysis provides additional evidence that SBI curricula are effective at improving students' conceptual understanding of statistical ideas post-course regardless student preparation. Further work is needed to better understand nuances of student improvement based on other student demographics, prior coursework, as well as instructor and institutional variables.

Student Performance in Curricula Centered on Simulation-Based Inference

“…As discussed in Tintle et al (2018), the instrument showed good reliability (Cronbach's alpha > 0.65), construct validity (e.g., stronger item-total correlations post-course than pre-course), and good predictive validity (moderate correlations with external measures of quantitative understanding (e.g., ACT score, r = 0.39); associations with positive attitudes and post-test scores (r > 0.25 for 5 of 6 SATS subscales). Additional details on reliability and validity for a single year of this sample (2016-2017) are provided in Tintle et al (2018).…”

“…Additionally, recent evidence at single institutions (Tintle et al, 2018, see also Figure 4) has suggested that students entering the course with weaker quantitative backgrounds may be among those that benefit the most from SBI curricula. Further research is needed to explore this trend across a wide range of institutions to confirm the generalizability of these findings and potentially different best practices curricularly or pedagogically when working with these groups.…”

Statistics education from a data‐centric perspective

“…The next essential ingredient in our teaching cocktail is simulation‐based inference (SBI). In addition to Cobb [6], who states that there is “no excuse not to teach SBI,” there is increasing evidence that SBI can improve statistical understanding (eg, [16,28,46]).…”

Use of Small-Scale Classroom Experiments to Inform Statistics (SBI) Pedagogy in Tertiary Classrooms

“…Previously our research on the effectiveness of SBI curricula has focused on measures of student learning in introductory statistics courses comparing SBI to non-SBI courses using pre/post course measures of conceptual understanding. Evidence is mounting that this approach not only improves student understanding of statistical inference but also improves understanding of the investigative process as a whole (e.g., Chance et al, 2022;Tintle et al, 2011Tintle et al, , 2018. However, are there "optimal" ways of introducing "simulation-based inference" to these students?…”