Progress Testing for Medical Students at the University of Auckland: Results from the First Year of Assessments

Abstract

BACKGROUND: Progress testing is a method of assessing longitudinal progress of students using a single best answer format pitched at the standard of a newly graduated doctor. AIM: To evaluate the results of the first year of summative progress testing at the University of Auckland for Years 2 and 4 in 2013. SUBJECTS: Two cohorts of medical students from Years 2 and 4 of the Medical Program. METHODS: A survey was administered to all involved students. Open text feedback was also sought. Psychometric data were collected on test performance, and indices of reliability and validity were calculated. RESULTS: The three tests showed increased mean scores over time. Reliability of the assessments was uniformly high. There was good concurrent validity. Students believe that progress testing assists in integrating science with clinical knowledge and improve learning. Year 4 students reported improved knowledge retention and deeper understanding. CONCLUSION: Progress testing has been successfully introduced into the Faculty for two separate year cohorts and results have met expectations. Other year cohorts will be added incrementally. RECOMMENDATION: Key success factors for introducing progress testing are partnership with an experienced university, multiple and iterative briefings with staff and students as well as demonstrating the usefulness of progress testing by providing students with detailed feedback on performance.

Does Progress Testing Violate the Principles of Constructive Alignment?

“…Interestingly, very little research has presented variance data from the results of the progress testing. In one study [8], the results of year 2 and 4 progress tests were depicted showing, contrary to what might be ideally expected (Fig. 1), that the variance was greater in year 4 than in year 2.…”

“…Advocates of progress testing present this deliberate misalignment as an advantage-they claim this is the decoupling between teaching and assessment activities which encourages deep learning. Furthermore, individual progress tests are intended to be assessments for learning rather than assessments of learning-in the sense that it is purported that testing can promote the element of curiosity and motivation for further learning, which will likely extend beyond the taught curriculum [8,20]. However, at least to date, evidence of deep learning engendered by progress testing remains equivocal [18] and almost entirely theoretical.…”

“…Progress testing has been described as an assessment process whereby students are examined in each year of their study using examination questions designed to represent the level of knowledge of the final qualifying examination [7][8][9][10]. Figure 1 shows a conceptual and schematic representation of the process of progress testing.…”

“…Progress tests appear to remove the high-stakes nature associated with traditional examinations and to reduce the stress of those being examined [8,9,13,17,18]. There is some evidence to suggest that progress tests, in comparison to traditional examinations, engender a more meaning-oriented study strategy rather than strategies based on a rote learning approach, although the evidence that it cultivates or motivates learning via more intrinsic or deeplearning mechanisms remains unclear [8,10,18]. There is also a belief that progress tests are more suited to students who work consistently and who are in medical schools that promote timely clinical contact [19].…”

The role of progress testing in tackling medical student anxiety

“…The prime motivation behind their development was to prevent test‐directed studying, whereby students memorise facts for immediate recall in end‐of‐unit examinations. Although this aim has appeared successful, by enforcing long‐term information retention, we suggest that there are several other benefits to this form of assessment.…”