The Australian Medical Schools Assessment Collaboration: benchmarking the preclinical performance of medical students

Abstract

Objectives: To report the level of participation of medical schools in the Australian Medical Schools Assessment Collaboration (AMSAC); and to measure differences in student performance related to medical school characteristics and implementation methods. Design: Retrospective analysis of data using the Rasch statistical model to correct for missing data and variability in item difficulty. Linear model analysis of variance was used to assess differences in student performance. Setting and participants: 6401 preclinical students from 13 medical schools that participated in AMSAC from 2011 to 2013. Main outcome measures: Rasch estimates of preclinical basic and clinical science knowledge. Results: Representation of Australian medical schools and students in AMSAC more than doubled between 2009 and 2013. In 2013 it included 12 of 19 medical schools and 68% of medical students. Graduate‐entry students scored higher than students entering straight from school. Students at large schools scored higher than students at small schools. Although the significance level was high (P < 0.001), the main effect sizes were small (4.5% and 2.3%, respectively). The time allowed per multiple choice question was not significantly associated with student performance. The effect on performance of multiple assessments compared with the test items as part of a single end‐of‐year examination was negligible. The variables investigated explain only 12% of the total variation in student performance. Conclusions: An increasing number of medical schools are participating in AMSAC to monitor student performance in preclinical sciences against an external benchmark. Medical school characteristics account for only a small part of overall variation in student performance. Student performance was not affected by the different methods of administering test items.

Motivating assessment item writers in medical programs: A qualitative study

“…In the UK, assessment of learning is an essential domain in the Professional Standards for medical educators from the Academy of Medical Educators [6]. In Australia, the University Teaching, Criteria and Standards framework cites assessment and feedback as one of seven criteria to be achieved by all who teach in higher education [7], and medical school assessment collaborations have been created to share and benchmark quality examination and clinical assessment items [8][9][10]. The majority (78%) of deans, directors of medical education and academic chairs believe that assessment is a 'very important' competency for clinician-educators [11].…”

Reconceptualising Health Professions Education in South Africa

“…However, academic achievement is only one of many factors, and may be a less useful predictor of subsequent clinical performance (Ferguson et al, 2002;Sommerville, 2012). There is local and international evidence that graduate entrants do better than students directly out of school, HE experience perhaps outweighing students' age (Ferguson et al, 2002;O'Mara et al, 2015;Sommerville, 2012). Patterson et al (2016) voice the concern that, as greater numbers of applicants score increasingly high marks, the ability to discriminate on the basis of prior academic achievement may be diminished.…”

Medical student satisfaction, coping and burnout in direct-entry versus graduate-entry programmes

“…Despite some differences in early (pre‐clinical) academic performance, there appears to be little difference in overall academic outcomes of DE versus GE students (e.g. medical school graduation rates, research degree completion and postgraduate training) . We hypothesised that there might be demonstrable differences in the non‐cognitive ‘psychosocial’ and ‘well‐being’ domains related to medical school, but there is a dearth of research comparing more than one dimension in more than one medical school.…”