Assessing the Reliability and Validity of the Mini—Clinical Evaluation Exercise for Internal Medicine Residency Training

Durning, Steven J.; Cation, Lannie J.; Markert, Ronald J.; Pangaro, Louis N.

doi:10.1097/00001888-200209000-00020

Cited by 161 publications

(155 citation statements)

References 7 publications

Supporting

Mentioning

132

Contrasting

Unclassified

Order By: Relevance

“…Though fairly new to medical training (Kogan et al 2009), a growing body of evidence on the validity and the reliability of formative assessment instruments is emerging (Durning et al 2002;Holmboe et al 2003;Wilkinson et al 2008;LeBlanc et al 2009). However, whereas in summative assessment validity and reliability are seen as dominant determinants of utility, in formative assessment, utility, defined as learning that results from the assessment process, is much more dependent on how stakeholders (trainees and clinical supervisors)…”

Section: Introductionmentioning

confidence: 99%

A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education

et al. 2013

View full text Add to dashboard Cite

Introduction: Recent changes in postgraduate medical training curricula usually encompass a shift towards more formative assessment, or assessment for learning. However, though theoretically well suited to postgraduate training, evidence is emerging that engaging in formative assessment in daily clinical practice is complex. Aim: We aimed to explore trainees' and supervisors' perceptions of what factors determine active engagement in formative assessment. Methods: Focus group study with postgraduate trainees and supervisors in obstetrics and gynaecology. Results: Three higher order themes emerged: individual perspectives on feedback, supportiveness of the learning environment and the credibility of feedback and/or feedback giver. Conclusion: Engaging in formative assessment with a genuine impact on learning is complex and quite a challenge to both trainees and supervisors. Individual perspectives on feedback, a supportive learning environment and credibility of feedback are all important in this process. Every one of these should be taken into account when the utility of formative assessment in postgraduate medical training is evaluated.

show abstract

Section: Introductionmentioning

confidence: 99%

A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education

et al. 2013

View full text Add to dashboard Cite

show abstract

“…Se acepta de forma general que el mini-CEX es váli-do y fi able cuando se usa en médicos en formación [5,6,11]. El primer estudio que analizó sus propiedades se publicó en 1995 [1].…”

Section: Factibilidad Y Propiedades Psicométricas En La Evaluación Deunclassified

“…Este enfoque puede reducir de seis a tres el número de evaluadores para alcanzar un coefi ciente de generalización igual o superior a 0,7. En lo que se refi ere a la validez, los valores del mini-CEX se correlacionan con los resultados de un examen escrito [6] y con los resultados de los exámenes orales del Royal College of Physicians and Surgeons de Canadá [15]. Por su parte, Al Ansari et al [16] analizaron la validez de constructo y de criterio del mini-CEX en 11 estudios que incluían datos empíricos de estas propiedades: ocho de ellos se referían a estudiantes posgraduados, y los otros tres, a estudiantes de medicina [17][18][19].…”

Section: Ferenchick Et Al [43]unclassified

See 1 more Smart Citation

La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicina

et al. 2015

View full text Add to dashboard Cite

La historia del Mini Clinical Evaluation Exercise(mini-CEX) empieza a principios de los años setenta, cuando la American Board of Internal Medicine (ABIM) estaba buscando una alternativa a las tradicionales evaluaciones orales de los residentes [1], como consecuencia de los problemas logísticos y de las limitadas propiedades psicométricas de este tipo de evaluaciones. La ABIM organizó un grupo de La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicinaJosep E. Baños, Carmen Gomar-Sancho, Elena Guardiola, Jordi Palés-ArgullósResumen. En los últimos años se recomienda el uso de instrumentos que evalúen la adquisición de habilidades clínicas en el contexto de la práctica clínica. Entre los instrumentos más recomendados se encuentra el Mini Clinical Evaluation Exercise (mini-CEX), desarrollado inicialmente para la evaluación de médicos residentes, y que ha demostrado una buena fi abilidad y validez. Aunque se ha utilizado en estudiantes de medicina, no se dispone de un análisis completo de la experiencia acumulada. Para la presente revisión se realizó en septiembre de 2014 la búsqueda de los artículos indizados en Medline, Embase, Biosis Previews y Current Contents. Se seleccionaron aquellos artículos que incluían los términos 'mini clinical evaluation exercise', 'mini-CEX' o 'mcex'. Tras la eliminación de los duplicados se obtuvo un total de 168 artículos. Se revisaron los que se referían al uso del mini-CEX en estudiantes de medicina (n = 34) y se analizaron a fondo aquellos que contenían datos empíricos (n = 21). Los estudios llevados a cabo en estudiantes de medicina confi rman que el mini-CEX es un instrumento útil y factible para evaluar las habilidades clínicas de los estudiantes de medicina y que posee propiedades psicométricas adecuadas en términos de validez y fi abilidad. Asimismo, es bien aceptado por estudiantes y tutores y puede ser usado tanto con fi nalidades formativas como sumativas. La importancia de dar un feedback adecuado y la concordancia de las puntuaciones de los tutores son los puntos más críticos en la implementación del mini-CEX en estudiantes de grado. Competing interests:None declared. © 2015 FEM Use of Mini Clinical Evaluation Exercise (mini-CEX) in medical studentsSummary. In recent years the use of work-based assessments has been increasingly recommended to evaluate clinical skills, as they allow for the evaluation of performance in the context of clinical practice. Among these tools, one of the most recommended is the Mini Clinical Evaluation Exercise (mini-CEX). This method was developed to be used in physicians-in-training and has good reliability and validity properties. Even when mini-CEX has also been used in medical students, a review of its value in undergraduates is lacking. This review is mainly based on a search of the articles indexed in Medline, Embase, Biosis Previews or Current Contents databases in September 2014. References were selected for articles which included the following terms: 'mini clinical evaluation exercise', 'mini-CEX'...

show abstract

“…For this reason, the body of research examining evaluations of residents cannot be assumed to be generalizable to medical students. [12][13][14][15][16] One previous study evaluating these effects did not attempt to resolve out the sources of the variability in the system and was possibly limited by the choice of statistical model. 4 Other studies examining for systematic error introduced by demographic data often yielded conflicting results, 3,4,[6][7][8]17,18 or were using more objective structured clinical encounters, or OSCEs.…”

Section: Introductionmentioning

confidence: 99%

Sources of variability in medical student evaluations on the internal medicine clinical rotation

et al. 2012

View full text Add to dashboard Cite

Objectives: To explore the sources of variability in evaluator ratings among third year medical students in the Internal Medicine clinical rotation. Also, to examine systematic effects and variability introduced by differences in the various student, evaluator, and evaluation settings. Methods: A multilevel model was used to estimate the amount of between-student, between-rater and raterstudent interaction variability present in the students' clinical evaluations in a third year internal medicine clinical rotation. Within this model, linear regression analysis was used to estimate the effect of variables on the students' numerical evaluation scores and the reliability of those scores. Results: A total of 2,747 evaluation surveys were collected from 389 evaluators on 373 students over 4.5 years. All surveys used a nine-point grading scale, and therefore all results are reported on this scale. The calculated betweenrater, between-student and rater-student interaction variance components were 0.50, 0.27 and 0.62, respectively. African American/Black students had lower scores than Caucasian students by 0.58 points (t=-3.28; P=0.001). No gender effects were noted. Conclusions: These between-rater and between-student variance components imply that the evaluator plays a larger role in the students' scores than the students themselves. The residual rater-student interaction variance was larger and did not change by accounting for the measured demographic variables. This implies there is significant variability in each rater-student interaction that remains unexplained. This could contribute to unreliability in the system, requiring that students receive between 8 and 17 clinical evaluations to achieve 80% reliability.

show abstract

Assessing the Reliability and Validity of the Mini—Clinical Evaluation Exercise for Internal Medicine Residency Training

Abstract: This study suggests that the mCEX is a feasible and reliable evaluation tool. The validity of the mCEX is supported by the strong correlations between mCEX scores and corresponding ABIM MEF scores as well as the ITE.

Cited by 161 publications

References 7 publications

A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education

A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education

La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicina

Sources of variability in medical student evaluations on the internal medicine clinical rotation

Contact Info

Product

Resources

About