Role of formative assessment in predicting academic success among GP registrars: a retrospective longitudinal study

Abstract

ObjectivesThe James Cook University General Practice Training (JCU GPT) programme’s internal formative exams were compared with the Royal Australian College of General Practitioners (RACGP) pre-entry exams to determine ability to predict final performance in the RACGP fellowship exams.DesignA retrospective longitudinal study.SettingGeneral Practice (GP) trainees enrolled between 2016 and 2019 at a Registered Training Organisation in regional Queensland, Australia.Participants376 GP trainees enrolled in the training programme.Exposure measuresThe pre-entry exams were Multiple-Mini Interviews (MMI), Situational Judgement Test (SJT) and Candidate Assessment and Applied Knowledge Test. The internal formative exams comprised multiple choice questions (MCQ1 and MCQ2), short answer questions, clinical skills and clinical reasoning.Primary outcome measureThe college exams were Applied Knowledge Test (AKT), Key Feature Problems (KFP) and Objective Structured Clinical Examination (OSCE).ResultsCorrelations (r), coefficients of determination (R2) and OR were used as parameters for estimating strength of relationship and precision of predictive accuracy. SJT and MMI were moderately (r=0.13 to 0.31) and MCQ1 and MCQ2 highly (r=0.37 to 0.53) correlated with all college exams (p<0.05 to p<0.01), with R2 ranging from 0.070 to 0.376. MCQ1 was predictive of failure in all college exams (AKT: OR=2.32, KFP: OR=3.99; OSCE: OR=3.46); while MCQ2 predicted failure in AKT (OR=2.83) and KFP (OR=3.15).ConclusionWe conclude that the internal MCQ formative exams predict performance in the RACGP fellowship exams. We propose that our formative assessment tools could be used as academic markers for early identification of potentially struggling trainees.

Insights from an updated personnel selection meta-analytic matrix: Revisiting general mental ability tests’ role in the validity–diversity trade-off.

“…We located 11 studies including a correlation between an interview and SJT. We excluded four of these studies because we could not determine whether or how much range restriction affected the sample (Heggarty et al, 2020; Husbands et al, 2018), whether the interview was structured (Husbands et al, 2018; Yingling et al, 2018), or the constructs measured in the SJT were very different from the SJT constructs in the rest of the samples (i.e., emotional management SJT in Knorr et al, 2018). This left seven studies which included eight independent samples, which we describe in the following.Morgeson et al (2005) reported an uncorrected correlation of .23 between a structured interview and SJT in a sample of 90 job incumbents.…”

On Educational Assessment Theory: A High-Level Discussion of Adolphe Quetelet, Platonism, and Ergodicity

“…As such, it is different in that there are no "high-stakes" test-taking or decision-making, but rather individual assessments provide feedback that then allows the learner to analyze "their own performance, formulate concrete learning goals and demonstrably attain them", which is opposed to the previously discussed assessment structure where the assessment is the evidence of learning or the lack thereof ( [84], p. 211). Further reading in this area should be directed to [85][86][87][88].…”

Knowledge Framework Construction, Question Bank Promotion and Case Collection Driving on Grade Point Average and the Medical Licensing Examination

“…There are many correlative or interrelated factors to the success of international students, both short-term and long-term, such as English pro ciency, nancial resources, coping strategies, self-e cacy, motivation, etc. Until now, there have been many investigations on practicable tactics to guide medical students to academic success [7][8][9]. However, studies on learning methods related to licensing exams are rare.…”