An investigation into the impact of question structure on the performance of first year physics undergraduate students at the University of Cambridge

Abstract

We describe a study of the impact of exam question structure on the performance of first year Natural Sciences physics undergraduates from the University of Cambridge. The results show conclusively that a student's performance improves when questions are scaffolded compared with university style questions. In a group of 77 female students we observe that the average exam mark increases by 13.4% for scaffolded questions, which corresponds to a 4.9 standard deviation effect. The equivalent observation for 236 male students is 9% (5.5 standard deviations). We also observe a correlation between exam performance and A2-level marks for UK students, and that students who receive their school education overseas, in a mixed gender environment, or at an independent school are more likely to receive a first class mark in the exam. These results suggest a mis-match between the problemsolving skills and assessment procedures between school and first year university and will provide key input into the future teaching and assessment of first year undergraduate physics students.

Impact of scaffolding and question structure on the gender gap

“…Of all exam questions, L exhibits one of the most significant differences in performance between genders, and H shows no significant difference. The scaffolding gains are comparable to those observed in [15] (13.4% for females, 8.8% for males). We conclude that scaffolding may play a role in reducing the gender gap in specific types of problems which were previously identified to contain a male bias, namely questions involving multi-dimensional context.…”

“…Although not as clear, the data does at first glance seem to support the conclusions of [15]. Male students outperform female students on the low scaffolding questions by 2.9% (p = .087), and by only 1.4% on the high scaffolding questions (p = .42).…”

“…Previous studies support the use of scaffolding in aiding students' learning and conceptual understanding in physics [16][17][18]. However, [15] is the first study to our knowledge to identify a dependence on gender. It is therefore important to verify these findings across institutions and student populations, prior to taking action towards any instructional reform.…”

Assessing student process skills and providing feedback to enhance learning in analytical chemistry

“…In many cases the scaffolding the researchers utilized included examples such as prompting students to complete more specific sub questions instead of one larger question. 154 The results from this study showed that scaffolding questions can benefit all undergraduate students but seemed to particularly have a positive influence on female students' exam performance. Students' expectations going into taking an exam can influence how students respond to the questions.…”

“…material is actually included on the exam, students may get an incorrect answer on the exam, not from lack of understanding about the content but because of their preconceived conceptions of the questions. 154 When a complex diagram or information was presented to students in another study conducted by Crip and Sweiry 156 , the students (sometimes incorrectly)…”