The implementation of a mobile problem-specific electronic CEX for assessing directly observed student–patient encounters

Ferenchick, Gary S.; Foreback, Jami; Towfiq, Basim; Kavanaugh, Kevin; Solomon, David; Mohmand, Asad

doi:10.3402/meo.v15i0.4276

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a second feasibility study, we subsequently demonstrated that this system could be used to deliver, and successfully implement, competency-based checklists for student assessment related to the CDIM curricular objectives using PDAs (Ferenchick et al, 2010). Data from these studies helped us determine that the distribution and use of curricular objectives and related assessment tools by students and faculty in our geographically dispersed medical school could be facilitated with just in time mobile technology.…”

Section: Discussionmentioning

confidence: 99%

“…Data from these studies helped us determine that the distribution and use of curricular objectives and related assessment tools by students and faculty in our geographically dispersed medical school could be facilitated with just in time mobile technology. Importantly, we also determined that students and preceptors valued the fact that the content and expected competencies were transparent and such transparency facilitated learner assessment (Ferenchick et al, 2010). However, technical issues with PDAs – such as lack of direct internet connection and the requirement to “synchronize” data from PDAs to the web using desktop computers – limited the practicality of PDA based assessment; a process that is not needed with contemporary internet enabled devices such as iPads, iPhones and other smartphones.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Using cloud-based mobile technology for assessment of competencies among medical students

Ferenchick

Solomon

2013

PeerJ

Self Cite

View full text Add to dashboard Cite

Valid, direct observation of medical student competency in clinical settings remains challenging and limits the opportunity to promote performance-based student advancement. The rationale for direct observation is to ascertain that students have acquired the core clinical competencies needed to care for patients. Too often student observation results in highly variable evaluations which are skewed by factors other than the student’s actual performance. Among the barriers to effective direct observation and assessment include the lack of effective tools and strategies for assuring that transparent standards are used for judging clinical competency in authentic clinical settings. We developed a web-based content management system under the name, Just in Time Medicine (JIT), to address many of these issues. The goals of JIT were fourfold: First, to create a self-service interface allowing faculty with average computing skills to author customizable content and criterion-based assessment tools displayable on internet enabled devices, including mobile devices; second, to create an assessment and feedback tool capable of capturing learner progress related to hundreds of clinical skills; third, to enable easy access and utilization of these tools by faculty for learner assessment in authentic clinical settings as a means of just in time faculty development; fourth, to create a permanent record of the trainees’ observed skills useful for both learner and program evaluation. From July 2010 through October 2012, we implemented a JIT enabled clinical evaluation exercise (CEX) among 367 third year internal medicine students. Observers (attending physicians and residents) performed CEX assessments using JIT to guide and document their observations, record their time observing and providing feedback to the students, and their overall satisfaction. Inter-rater reliability and validity were assessed with 17 observers who viewed six videotaped student-patient encounters and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient OSCE exams. A total of 3567 CEXs were completed by 516 observers. The average number of evaluations per student was 9.7 (±1.8 SD) and the average number of CEXs completed per observer was 6.9 (±15.8 SD). Observers spent less than 10 min on 43–50% of the CEXs and 68.6% on feedback sessions. A majority of observers (92%) reported satisfaction with the CEX. Inter-rater reliability was measured at 0.69 among all observers viewing the videotapes and these ratings adequately discriminated competent from non-competent performance. The measured CEX grades correlated with subsequent student performance on an end-of-year OSCE. We conclude that the use of JIT is feasible in capturing discrete clinical performance data with a high degree of user satisfaction. Our embedded checklists had adequate inter-rater reliability and concurrent and predictive validity.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Using cloud-based mobile technology for assessment of competencies among medical students

Ferenchick

Solomon

2013

PeerJ

Self Cite

View full text Add to dashboard Cite

show abstract

“…Ferenchick et al [9] suggest flexible assessment tools are needed during routine clinical workflow, and a number of authors propose that mobile technologies offer potential options for more flexible modes of workplace-based assessments [9][10][11]. Internet-enabled mobile devices such as smartphones and tablets have become ubiquitous, and are being used with increasing frequency in medical education.…”

Section: Background and Need For Innovationmentioning

confidence: 99%

Using mobile technology in assessment of entrustable professional activities in undergraduate medical education

Duggan¹,

Curran²,

Fairbridge³

et al. 2020

Perspect Med Educ

View full text Add to dashboard Cite

Background The adoption of competency-based medical education requires objective assessments of a learner’s capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum. Approach A paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017–2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience. Evaluation The mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students’ scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved. Reflection Our preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.

show abstract

“…Ferenchick et al [42] también describieron su experiencia con el empleo de la PDA para mostrar los listados de competencias en el mini-CEX con el objeto de facilitar la observación directa y la evaluación de los estudiantes de medicina. Su principal conclusión fue que el método es factible y tiene un impacto positivo en la capacidad del evaluador para proporcionar feedback y para realizar la valoración.…”

Section: Aplicación De La Tecnología Y Otros Ejemplos De Mejoras En Eunclassified

La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicina

et al. 2015

View full text Add to dashboard Cite

La historia del Mini Clinical Evaluation Exercise(mini-CEX) empieza a principios de los años setenta, cuando la American Board of Internal Medicine (ABIM) estaba buscando una alternativa a las tradicionales evaluaciones orales de los residentes [1], como consecuencia de los problemas logísticos y de las limitadas propiedades psicométricas de este tipo de evaluaciones. La ABIM organizó un grupo de La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicinaJosep E. Baños, Carmen Gomar-Sancho, Elena Guardiola, Jordi Palés-ArgullósResumen. En los últimos años se recomienda el uso de instrumentos que evalúen la adquisición de habilidades clínicas en el contexto de la práctica clínica. Entre los instrumentos más recomendados se encuentra el Mini Clinical Evaluation Exercise (mini-CEX), desarrollado inicialmente para la evaluación de médicos residentes, y que ha demostrado una buena fi abilidad y validez. Aunque se ha utilizado en estudiantes de medicina, no se dispone de un análisis completo de la experiencia acumulada. Para la presente revisión se realizó en septiembre de 2014 la búsqueda de los artículos indizados en Medline, Embase, Biosis Previews y Current Contents. Se seleccionaron aquellos artículos que incluían los términos 'mini clinical evaluation exercise', 'mini-CEX' o 'mcex'. Tras la eliminación de los duplicados se obtuvo un total de 168 artículos. Se revisaron los que se referían al uso del mini-CEX en estudiantes de medicina (n = 34) y se analizaron a fondo aquellos que contenían datos empíricos (n = 21). Los estudios llevados a cabo en estudiantes de medicina confi rman que el mini-CEX es un instrumento útil y factible para evaluar las habilidades clínicas de los estudiantes de medicina y que posee propiedades psicométricas adecuadas en términos de validez y fi abilidad. Asimismo, es bien aceptado por estudiantes y tutores y puede ser usado tanto con fi nalidades formativas como sumativas. La importancia de dar un feedback adecuado y la concordancia de las puntuaciones de los tutores son los puntos más críticos en la implementación del mini-CEX en estudiantes de grado. Competing interests:None declared. © 2015 FEM Use of Mini Clinical Evaluation Exercise (mini-CEX) in medical studentsSummary. In recent years the use of work-based assessments has been increasingly recommended to evaluate clinical skills, as they allow for the evaluation of performance in the context of clinical practice. Among these tools, one of the most recommended is the Mini Clinical Evaluation Exercise (mini-CEX). This method was developed to be used in physicians-in-training and has good reliability and validity properties. Even when mini-CEX has also been used in medical students, a review of its value in undergraduates is lacking. This review is mainly based on a search of the articles indexed in Medline, Embase, Biosis Previews or Current Contents databases in September 2014. References were selected for articles which included the following terms: 'mini clinical evaluation exercise', 'mini-CEX'...

show abstract

The implementation of a mobile problem-specific electronic CEX for assessing directly observed student–patient encounters

Cited by 7 publications

References 11 publications

Using cloud-based mobile technology for assessment of competencies among medical students

Using cloud-based mobile technology for assessment of competencies among medical students

Using mobile technology in assessment of entrustable professional activities in undergraduate medical education

La utilización del Mini Clinical Evaluation Exercise (mini-CEX) en estudiantes de medicina

Contact Info

Product

Resources

About