Demographic gaps or preparation gaps?: The large impact of incoming preparation on performance of students in introductory physics

Abstract

We have studied the impact of incoming preparation and demographic variables on student performance on the final exam in the standard introductory calculus-based mechanics course at three different institutions. Multivariable regression analysis was used to examine the extent to which exam scores can be predicted by a variety of variables that are available to most faculty and departments. The results are surprisingly consistent across the institutions, with only math SAT or ACT scores and concept inventory prescores having predictive power. They explain 20%-30% of the variation in student exam performance in all three cases. In all cases, although there appear to be gaps in exam performance if one considers only demographic variables (gender, underrepresented minority, first generation), once these two proxies of incoming preparation are controlled for, there is no longer a demographic gap. There is only a preparation gap that applies equally across the entire student population. This work shows that to properly understand differences in student performance, it is important to do statistical analyses that take multiple variables into account, covering both subject-specific and general preparation. Course designs and teaching better matched to the incoming student preparation will likely eliminate performance gaps across demographic groups, while also improving the success of all students.

What do AP physics courses teach and the AP physics exam measure?

“…As we show in Ref. [20], concept inventory prescores explain a large fraction of students' performance in physics even after controlling for general measures of incoming preparation and demographic variables [20]. Furthermore, we see that scores on the AP physics exam are too weakly correlated with students' performance on the FMCE or final exam in the Physics 1 course to provide any confidence that it is a suitable substitute for a university introductory mechanics course.…”

“…As the FMCE involves no math, we interpret the math SAT score as a proxy for their general level of college science, technology, engineering, and mathematics (STEM) preparation. In other work [20] we have seen that math SAT scores significantly correlate with a variety of physics specific outcomes, while other SAT scores do not. 4We use multiple linear regression to simultaneously examine the relative contributions of how both taking AP physics (or a college-level physics class) and math SAT score contribute to a student's predicted performance on the FMCE.…”

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

“…As such, some of the complexity in Table I results from these items. If items 5,6,15,20,21,33,35,37, and 39 are eliminated from the analysis, the structure of Table I simplifies substantially to produce Table II. The communities in Table II are shown graphically in Fig.…”

“…"Coin Toss -Acceleration" items (items [27][28][29] ask students to select the acceleration of a coin tossed in the air. "Newton III" items (items [30][31][32][33][34][35][36][37][38][39] ask students about the forces during a variety of interactions between cars and trucks. "Velocity Graph" items (items [40][41][42][43] ask students to select the graph which correctly represents the velocity of a toy car moving on a horizontal surface.…”

“…Researchers have explored various factors that could explain the differences between men and women on the FMCE. For example, differences in academic backgrounds and preparation, measured by FMCE pretest and math placement exam scores, have been shown to explain much of the gender gap on the FMCE post-test [34,35]. Studies have also investigated the impact of interactive-engagement on the overall gender gap.…”