Marie Thoresen er spesialist i indremedisin og i lungesykdommer og har mastergrad i medisinsk utdanning. Hun er seniorrådgiver i Helsedirektoratet, der hun arbeider med spesialistutdanningen for leger. Forfa eren har fylt ut ICMJE-skjemaet og oppgir ingen interessekonflikter. Lederartikkelen representerer hennes egne synspunkter, og er ikke Helsedirektoratets offisielle standpunkt.

No abstract

BackgroundUntil 2008, Objective Structured Clinical Examinations (OSCE) were well researched, laborious and costly paper based method of exam delivery restricting international comparison. Cross-institutional comparison of OSCE Quality Assurance in Europe has never been done and due to wide spread electronic assessment analysis is now available.MethodsTwenty educational institutions across Europe using an electronic OSCE Management Information System where invited of which 8 confirmed to join a mutual comparison of Quality Assurance outcome. Two theories evaluate the quality of the observed test scores, the Classical psychometric theory (Cronbach’s alpha) and the Generalizability theory. Outcomes for both were compared for all universities including the Standard Error of Measurement (SEM) as well as cut-scores, Pass/Fail score and Global Rating Scores, Cronbach’s Alpha and related SEM (68% and 95% CI) and G-theory Coefficients with related absolute and relative SEM (68% and 95% CI). ResultsOutcomes differ between participating universities and observed marks contradict global rating of fail, borderline and excellent performance. G-theory coefficients and Standard Error of Measurement were lower and smaller compared to the classical approach using Cronbach’s Alpha as measure of reliability. The Classical psychometric based SEM varies from 2.8% to 11.2% respectively whereas the 95% CI equivalent varied from 9.2% up to 22% (on a 0 - 100% scale). The relative SEM from G-theory analysis varied from 3.1% to 7.0% for criterion-referenced marks, and the absolute SEM for norm-referenced marks varied from 3.8% to 7.8% respectively. The 95% CI around the relative and absolute SEMs values varied from 7.3% to 15.3%. More students failed the examination if the 95% CI is applied to the observed scores.ConclusionTo protect society and to improve educational decision making, the Standard Error of Measurement and associated confidence intervals needs to be embedded in EU assessment strategies to rule out ‘false positive Pass decisions’.