IntroductionMini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences.MethodsWe searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr’s adaptation of Kirkpatrick’s four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher’s exact test for count data.ResultsWe identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners’ reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees’ performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, “quality” and “participant responsiveness”, to be associated with the educational impact.ConclusionsDespite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.

No abstract

Our model of influencing factors might help to further improve the use of Mini-CEX and DOPS and serve as basis for future research.

Multiple true-false (MTF) items are a widely used supplement to the commonly used single-best answer (Type A) multiple choice format. However, an optimal scoring algorithm for MTF items has not yet been established, as existing studies yielded conflicting results. Therefore, this study analyzes two questions: What is the optimal scoring algorithm for MTF items regarding reliability, difficulty index and item discrimination? How do the psychometric characteristics of different scoring algorithms compare to those of Type A questions used in the same exams? We used data from 37 medical exams conducted in 2015 (998 MTF and 2163 Type A items overall). Using repeated measures analyses of variance (rANOVA), we compared reliability, difficulty and item discrimination of different scoring algorithms for MTF with four answer options and Type A. Scoring algorithms for MTF were dichotomous scoring (DS) and two partial credit scoring algorithms, PS where examinees receive half a point if more than half of true/false ratings were marked correctly and one point if all were marked correctly, and PS where examinees receive a quarter of a point for every correct true/false rating. The two partial scoring algorithms showed significantly higher reliabilities (α = 0.75; α = 0.75; α = 0.70, α = 0.72), which corresponds to fewer items needed for a reliability of 0.8 (n = 74; n = 75; n = 103, n = 87), and higher discrimination indices (r = 0.33; r = 0.33; r = 0.30; r = 0.28) than dichotomous scoring and Type A. Items scored with DS tend to be difficult (p = 0.50), whereas items scored with PS become easy (p = 0.82). PS and Type A cover the whole range, from easy to difficult items (p = 0.66; p = 0.73). Partial credit scoring leads to better psychometric results than dichotomous scoring. PS covers the range from easy to difficult items better than PS. Therefore, for scoring MTF, we suggest using PS.

Medical education has a long tradition of using various patient representations in teaching and assessment. With this literature review we aim, first, to provide an overview of the most important patient representations used to teach and assess clinical skills, considering in particular "summative exams" that have a pass or fail outcome; second, to provide arguments for choosing certain patient representations; and third, to show the advantages and limitations of different patient representations, especially simulated patients (SPs) and real patients (RPs).We conclude that in order to select the right patient representations in clinical teaching and/or assessment, a number of perspectives must be considered: (i) the learning goals, aligned with the stage of study, (ii) the correspond-ing requirements of the clinical task itself (e.g., performing a phlebotomy or a communication task), (iii) the level of authenticity required and (iv) the resources needed, taking patient safety and feasibility into consideration.