Electronic Rating of Objective Structured Clinical Examinations: Mobile Digital Forms Beat Paper and Pencil Checklists in a Comparative Study

Schmitz, Felix; Zimmermann, Philippe; Gaunt, Kevin; Stolze, Markus; Schär, Sissel Guttormsen

doi:10.1007/978-3-642-25364-5_35

Cited by 15 publications

(19 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition, the electronic system was highly rated by the assessors, who found it less invasive and reported that they had more time to observe the students and permitted greater observation of the students when using the paper assessment. Schmitz et al (2011) highlight a number of advantages to use an electronic handheld device to assess OSCEs including, speed of data gathering, simplicity of data evaluation and fast automatic feedback (Schmitz et al, 2011). Segall et al (2005) support computer based assessment suggesting that grading is more accurate, feedback is immediate, security is enhanced and less time is spent by instructors on grading and data entry (Segall, et al, 2005).…”

Section: Introductionmentioning

confidence: 98%

Back to the future: An online OSCE Management Information System for nursing OSCEs

Meskell

Burke

Kropmans

et al. 2015

Nurse Education Today

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 98%

Back to the future: An online OSCE Management Information System for nursing OSCEs

Meskell

Burke

Kropmans

et al. 2015

Nurse Education Today

View full text Add to dashboard Cite

“…In addition, the electronic system was highly rated by the assessors, who found it less invasive and reported they had more time to observe the students and permitted greater observation of the students when using the paper assessment. Schmitz [11] highlights a number of advantages to use an electronic handheld device to assess OSCEs including, speed of data gathering, simplicity of data evaluation and fast automatic feedback [11]. Segall et al [10] support computer based assessment suggesting that grading is more accurate, feedback is immediate, security is enhanced and less time is spent by instructors on grading and data entry [10].…”

Section: Resultsmentioning

confidence: 99%

Back to the Future: Electronic Marking of Objective Structured Clinical Examinations and Admission Interviews Using an Online Management Information System in Schools of Health Sciences

Kropmans¹

2015

J Health Med Informat

View full text Add to dashboard Cite

Background: The Objective Structured Clinical Examination (OSCE) and Multi Mini Interviews (MMI) are established tools in the repertoire of clinical assessment methods in Schools of Medicine and Health Sciences worldwide. The use of OSCEs facilitates the assessment of psychomotor skills as well as knowledge and attitudes. Identified benefits of OSCE assessment include development of students' confidence in their clinical skills and preparation for clinical practice. However, a number of challenges exist with the traditional paper methodology, including documentation errors and inadequate student feedback, electronic assessment is therefore new future.Objectives: To explore electronic OSCE delivery and evaluate the benefits of using an electronic OSCE management system. Design: A pilot study was conducted using electronic software in the management of a five station OSCE assessment with a cohort of first year undergraduate medical students delivered over two consecutive years (n=383) in one higher education institution in Ireland.Methods: All OSCE documentation was converted to electronic format. Assessors were trained in the use of the OSCE management software package and laptops were procured to facilitate electronic management of the OSCE assessment. Following the OSCE assessment, assessors were invited to evaluate the experience.Results: Electronic software facilitated the storage and analysis of overall group and individual results thereby offering considerable time savings. Submission of electronic forms was allowed only when fully completed thus removing the potential for missing data. Conclusions:Analysis of results highlights issues around inter-rater reliability and validity of measurement tools. Regression analysis, as a standard setting method, increases fairness of result calculations as compared to static cutoff scores.

show abstract

“…These findings are consistent with Schmitz and colleagues who found the software to have a usability rating of 6.5 on a 7-point Likert scale, a strong examiner preference over paper-based checklists and that the use of tablet devices required lower levels of mental effort by examiners. 4 …”

Section: Discussionmentioning

confidence: 99%

“…4 Addressing these tasks after the completion of an OSCE consumes an appreciable amount of academic and administrative staff time, especially when coordination is necessarily spread across distributed examination centres. Schmitz and colleagues concluded that up to sixty percent of examination papers have missing or incomplete results, though in our experience this figure was much lower.…”

Section: Introductionmentioning

confidence: 99%

“…Schmitz and colleagues concluded that up to sixty percent of examination papers have missing or incomplete results, though in our experience this figure was much lower. 4 However minimal, any missing data requires further clarification. This can be difficult to achieve and is a threat to the validity of the assessment item for an individual student or a whole cohort if the missing data rate is high.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Disruptive Innovation: successful implementation of an electronic marking in an Objective Structured Clinical Examinations – a single institution case study

Fraser¹,

Lane²

2017

MedEdPublish

View full text Add to dashboard Cite

Problem: Objective Structured Clinical Examinations are commonly included in the suite of assessment tasks in health professional programs. Conducting these assessments using paper-based methods has resulted in challenges that have been identified across the range of disciplines. These challenges include laborious administrative processes, missing assessment data, delays in data analysis and result release, which may limit the capacity to provide feedback to students. Intervention:The implementation of an electronic marking system was piloted in an attempt to address these issues. The pilot included the identification and testing of the software, evaluation of its impacts on the conduct and outcomes of the assessment and the identified challenges. The outcomes were considered through the lens of disruptive innovation.Context: This medical program has an annual intake of more than 500 students with learning, teaching and assessment occurring at multiple distributed campuses in metropolitan, regional and rural areas of Queensland, Australia, and the United States of America. This results in a complex and resource-intensive assessment and data management process. The identified challenges were found to be compounded when the examination was delivered concurrently at several examination centres to a large cohort.Outcomes: The introduction of an electronic marking system was feasible to implement across multiple examination sites and to a large cohort, and was found to be more reliable and efficient compared with traditional paper-based assessment systems. Examiner feedback regarding accessibility and acceptability of electronic marking on tablet devices for clinical assessments was favourable with enhancements to qualitative feedback and no adverse impact on student results being identified.Lessons Learned: The utilization of an electronic marking system in Objective Structured Clinical Examinations was found to be consistent with both disruptive and sustainable innovation. The opportunity to provide immediate feedback was consistent with the definition of a disruptive innovation and improved data management and processing outcomes consistent with a sustainable innovation. Furthermore, the use of electronic marking systems technology is likely to be transferable to other competency and workplace-based assessments.

show abstract

Electronic Rating of Objective Structured Clinical Examinations: Mobile Digital Forms Beat Paper and Pencil Checklists in a Comparative Study

Cited by 15 publications

References 10 publications

Back to the future: An online OSCE Management Information System for nursing OSCEs

Back to the future: An online OSCE Management Information System for nursing OSCEs

Back to the Future: Electronic Marking of Objective Structured Clinical Examinations and Admission Interviews Using an Online Management Information System in Schools of Health Sciences

Disruptive Innovation: successful implementation of an electronic marking in an Objective Structured Clinical Examinations – a single institution case study

Contact Info

Product

Resources

About