Twelve tips for OSCE-style Tele-assessment

Abstract

The Australian College of Rural and Remote Medicine (ACRRM) has developed a flexible 'tele-assessment' approach to the delivery of its assessment modalities. Candidates can sit their examination remotely, close to their place of practice, which reduces the need for rural doctors-both candidates and examiners-to leave their communities for the purpose of assessment. A major component of the assessment process is the Structured Assessment using Multiple Patient Scenarios (StAMPS) examination, which blends the formats of an Objective Structured Clinical Examination (OSCE) and a traditional viva vocè examination. It is a high-stakes assessment, that was designed to be academically rigorous, flexible, valid, reliable, and fair. Since 2008 ACRRM has provided a videoconferencing option to candidates for their StAMPS examination allowing them to remain in or near their home location, while the examiners meet a central location. Travel restrictions due to the SARS-CoV-2 pandemic meant for the first time both candidates AND examiners participated in StAMPS via videoconference. ACRRM conducted an online StAMPS assessment using videoconferencing technology for 65 candidates in mid-May 2020, with all candidates, examiners and support staff remaining in or near their home communities. These Twelve Tips outline some of the experience gained in providing tele-assessment over the past twelve years.

Eleven ways to get a grip on the implementation of remote administration of high-stakes assessments

“…Providing practice opportunities for candidates, examiners and standardized patients to familiarize themselves with the technology can help to reduce the anxiety, 15,16 and favors a smoother administration because everyone involved knows the process and the technology. 17 It seems that just being exposed to the platform reduces the anxiety of all users (examinees, examiners and standardized patients). 16,17 This applies for both written and performance-based assessment.…”

“…17 It seems that just being exposed to the platform reduces the anxiety of all users (examinees, examiners and standardized patients). 16,17 This applies for both written and performance-based assessment. When simulation or practice sessions are not feasible, one could consider podcasts or step-by-step demonstrations to prepare candidates, standardized patients, and examiners.…”

“…In the context of Objective Structured Clinical Examinations (OSCEs) for example, one might consider using a different approach to candidates moving from room to room. Lewandowski et al 17 and Ryan et al, 18 for example, favored an approach where examiners moved from candidate to candidate. These movements -from room to room-were facilitated by resource dedicated to the management of examinee and examiner location.…”

'Stability' of Assessment: Extending the Utility Equation

“…A recent paper describes Twelve Tips for delivering high-stakes OSCE-style examinations via video teleassessment, based on ACRRM's extensive experience since 2008, both in tele-assessment in the rural context and with subsequent adaptations for the COVID environment (Lewandowski et al, 2020).…”

“…They offered a number of recommendations on risk management, including a formal risk management strategy and concluded, 'With current readily available technologies many assessments that traditionally were performed in person can now be performed via tele-assessment improving convenience, reducing travel time and costs, and allowing adaptation to necessities and impediments outside of the examination itself.' (Lewandowski et al, 2020) Nevertheless, exams can fail, both due to IT and non-IT factors. While the ACRRM StAMPS exam illustrates some Sen Gupta T, Wong E, Doshi D, Hays R MedEdPublish https://doi.org/10.15694/mep.2021.000155.1…”