Creating a learner performance dashboard for programmatic assessment

Abstract

Editor's note: Programmatic assessment is a process that is gaining traction in medical and health professional education. It is an approach that aligns with competency-based education. Routine and longitudinal data are collected about learners' competence, and their progress is regularly reviewed and analysed. Typically, a variety of assessment methods are used with the overall aim of allowing both learners and educators to gain an understanding of the extent of learning and to maximise feedback for optimal educational impact (assessment for learning). Eventually the process may be used for high-stakes decisions at the end of a programme or phase of study (assessment of learning). In this toolbox article the authors describe how they have developed a method of visually representing medical student attainment during programmatic assessment on a dashboard. This development required input from educators, faculty members and information technology staff. The graphs obtained can help students and staff to see how a student is performing over time. The authors share feedback on the approach from learners and faculty members and provide recommendations for readers who may wish to develop something similar.

Towards a flexible and personalised development of veterinarians and veterinary nurses working in a companion animal referral care setting

“…It focuses on optimising learning through formative assessment ( learning function of assessment ), while simultaneously gathering valuable information that can be used for summative high‐stakes decision‐making about learner progression ( decision function of assessment ) (Van der Vleuten & Schuwirth, 2019 ; Van der Vleuten et al., 2012 ). Programmatic assessment is becoming increasingly popular in both medical and veterinary education (Bok et al., 2018 ; Santen et al., 2020 ; Van der Vleuten et al., 2012 ). It aligns with CBE and collects routine and longitudinal data about knowledge, skills, and competence development.…”

“…Progress is regularly reviewed and analysed. Assessment data are collected, stored, and presented clearly to provide meaning for learners and those responsible for judging competency and progression (Friedman et al., 2016 ; Monrad et al., 2019 ; Santen et al., 2020 ; Spickard et al., 2016 ).…”

“…This repository should be owned by each individual to steer self‐directed learning strategies and serve as a reminder to reflect on one's own developing competence and EPA entrustment decisions (Bok et al., 2018 ; Ilgen et al., 2021 ). It could be used to systematically document feedback and self‐reflection, provide a visualisation of learner development over time, and offer insight into the candidate's strengths and weaknesses (Favier et al., 2019 ; Santen et al., 2020 ). Currently, we are experimenting with a prototype of a digital repository that meets these requirements.…”

The critical role of infrastructure and organizational culture in implementing competency-based education and individualized pathways in undergraduate medical education

“…Data capture, organization, and visualization tools are necessary to make performance evidence interpretable and actionable (Boscardin et al 2018). Consortium members shared key features of developing informatics platforms and challenges encountered (Santen et al 2020). Although competency milestones rely on narrative descriptors, a common pitfall of dashboards noted in implementation was conversion of narratives to numeric data for reporting and display.…”