An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education

Abstract

BackgroundWe evaluated a newly designed electronic portfolio (e-Portfolio) that provided quantitative evaluation of surgical skills. Medical students at the University of Seville used the e-Portfolio on a voluntary basis for evaluation of their performance in undergraduate surgical subjects.MethodsOur new web-based e-Portfolio was designed to evaluate surgical practical knowledge and skills targets. Students recorded each activity on a form, attached evidence, and added their reflections. Students self-assessed their practical knowledge using qualitative criteria (yes/no), and graded their skills according to complexity (basic/advanced) and participation (observer/assistant/independent). A numerical value was assigned to each activity, and the values of all activities were summated to obtain the total score. The application automatically displayed quantitative feedback. We performed qualitative evaluation of the perceived usefulness of the e-Portfolio and quantitative evaluation of the targets achieved.ResultsThirty-seven of 112 students (33%) used the e-Portfolio, of which 87% reported that they understood the methodology of the portfolio. All students reported an improved understanding of their learning objectives resulting from the numerical visualization of progress, all students reported that the quantitative feedback encouraged their learning, and 79% of students felt that their teachers were more available because they were using the e-Portfolio. Only 51.3% of students reported that the reflective aspects of learning were useful. Individual students achieved a maximum of 65% of the total targets and 87% of the skills targets. The mean total score was 345 ± 38 points. For basic skills, 92% of students achieved the maximum score for participation as an independent operator, and all achieved the maximum scores for participation as an observer and assistant. For complex skills, 62% of students achieved the maximum score for participation as an independent operator, and 98% achieved the maximum scores for participation as an observer or assistant.ConclusionsMedical students reported that use of an electronic portfolio that provided quantitative feedback on their progress was useful when the number and complexity of targets were appropriate, but not when the portfolio offered only formative evaluations based on reflection. Students felt that use of the e-Portfolio guided their learning process by indicating knowledge gaps to themselves and teachers.

Review of e-portfolio requirements and methodology

“…Several studies [7,9,11,22,23] have advocated that e-portfolios are particularly useful as a record of experience and promotes independent reflective learning, to think back about those experiences. The record can be in the form of a logbook or a diary of day to day work experiences, [5,6] which is especially important in the highly skills-dependent healthcare profession where the learning takes place in a working clinical environment as opposed to a classroom teaching basis.…”

“…The record can be in the form of a logbook or a diary of day to day work experiences, [5,6] which is especially important in the highly skills-dependent healthcare profession where the learning takes place in a working clinical environment as opposed to a classroom teaching basis. [24] Several studies [9,11,13] have also found that trainees prefer an electronic interface of managing a portfolio, stating it to be better suited to the active way of work in a clinical environment. In the UK, electronic portfolio interface is now readily available to clinicians and trainee doctors by most institutes and Royal Colleges, [5,6] and even generic ones are freely available by various organisation that cater to the medical profession as a whole, for example the BMA.…”

“…[10][11][12][13] It can be as simple as "read up NICE guidelines in managing unstable angina" or something that requires more planning such as "arrange a session in the GUM clinic and explore ethical considerations in sexual health".…”

“…[9] Various medical schools have also developed their own portfolio databases for students to upload their work, log reflections and feedback [7,10,11] as a means of an early introduction to what will be a life-long practice of reflective learning throughout the students' careers in accordance with the GMC's guidance Tomorrow's Doctors (2009). [12] As much as the e-portfolio is important, various studies have shown that it is met with mixed responses and perceptions from junior doctors and medical students.…”

OSCE para Competências de Comunicação Clínica e Profissionalismo: Relato de Experiência e Meta-Avaliação

“…Entre as competências essenciais na prática clíni-ca, temos a comunicação clínica e o profissionalismo, para os quais o OSCE tem sido um método utlizado com certa frequência e com bons resultados em alguns estudos [17][18][19][20] . Entre suas aplicações, tem a capacidade de avaliar a anamnese, o exame físico e a interpretação de resultados clínicos, com destaque para a comunicação efetiva 9,13 .…”