Disruptive Innovation: successful implementation of an electronic marking in an Objective Structured Clinical Examinations – a single institution case study

Abstract

Problem: Objective Structured Clinical Examinations are commonly included in the suite of assessment tasks in health professional programs. Conducting these assessments using paper-based methods has resulted in challenges that have been identified across the range of disciplines. These challenges include laborious administrative processes, missing assessment data, delays in data analysis and result release, which may limit the capacity to provide feedback to students. Intervention:The implementation of an electronic marking system was piloted in an attempt to address these issues. The pilot included the identification and testing of the software, evaluation of its impacts on the conduct and outcomes of the assessment and the identified challenges. The outcomes were considered through the lens of disruptive innovation.Context: This medical program has an annual intake of more than 500 students with learning, teaching and assessment occurring at multiple distributed campuses in metropolitan, regional and rural areas of Queensland, Australia, and the United States of America. This results in a complex and resource-intensive assessment and data management process. The identified challenges were found to be compounded when the examination was delivered concurrently at several examination centres to a large cohort.Outcomes: The introduction of an electronic marking system was feasible to implement across multiple examination sites and to a large cohort, and was found to be more reliable and efficient compared with traditional paper-based assessment systems. Examiner feedback regarding accessibility and acceptability of electronic marking on tablet devices for clinical assessments was favourable with enhancements to qualitative feedback and no adverse impact on student results being identified.Lessons Learned: The utilization of an electronic marking system in Objective Structured Clinical Examinations was found to be consistent with both disruptive and sustainable innovation. The opportunity to provide immediate feedback was consistent with the definition of a disruptive innovation and improved data management and processing outcomes consistent with a sustainable innovation. Furthermore, the use of electronic marking systems technology is likely to be transferable to other competency and workplace-based assessments.