There is a substantial body of work in physics education looking at gender disparities in physics. Recent work has linked gender disparities in college physics course performance to disparities in high school physics preparation, but to our knowledge, the origin of the disparity in high school physics preparation is still underexplored. In a select sample, we found that women on average had lower force and motion conceptual evaluation (FMCE) pre-scores (the FMCE is a short conceptual assessment of Newton’s laws), and FMCE pre-score entirely mediated the effects of high school preparation and social-psychological factors on exam performance. The gender gap in FMCE pre-scores could not be explained by differences in the number of physics courses taken in high school. Instead, we find that the gender gap in the FMCE is partially explained by female students’ higher levels of general test anxiety. We hypothesize that the format of the FMCE, a timed assessment, triggers stereotype threat in female students despite being a low-stakes assessment. Therefore, instructors and researchers should take care in interpreting the results of such concept inventory scores and should re-think the way they assess understanding of physics concepts. Results of this work aligned with previous findings on gender disparity in timed exams call upon investigating gender equitable assessment formats for evaluating physics knowledge to replace timed assessments, either high or low stakes.