Evaluating Objective Structured Clinical Assessment (OSCA) in undergraduate nursing

Abstract

The aims of this mixed methods study were to gain insight into how individual assessors determine an Objective Structured Clinical Assessment (OSCA) result for undergraduate nursing students and identify whether individual assessor perceptions and professional characteristics have an impact on students' results. Results from 25 participants showed that although less than half (44%) of the participants were teaching in the course that they were assessing, the participants were highly experienced clinicians and nearly threequarters (72%) had completed formal teaching qualifications. There were wide variations in pass rates (16.7-90%) between assessors. The widest disparity was observed between assessors with and those without critical care experience (66% versus 39%), as well as assessors who were teaching the course and those who were not (68% versus 49%). Qualitative analysis revealed three dominant themes within participants' transcripts. The themes focused on determining student safety, and the use of personal perceptions and clinical experience to determine competency. Findings indicate that assessors' individual perceptions and clinical experience have the potential to influence and determine undergraduate nursing students OSCA results. Development of criteria standards and objective assessment may be enhanced by greater involvement of assessors and thorough education and training within the context of student assessments. KeywordsObjective structured clinical assessment, undergraduate nursing, nursing practice, assessment, competence AbstractThe aims of this mixed methods study were to gain insight into how individual assessors determine an Objective Structured Clinical Assessment (OSCA) result for undergraduate nursing students and identify whether individual assessor perceptions and professional characteristics have an impact on students' results. Results from 25 participants showed that although less than half (44%) of the participants were teaching in the course that they were assessing, the participants were highly experienced clinicians and nearly three-quarters (72%) had completed formal teaching qualifications. There were wide variations in pass rates (16.7-90%) between assessors. The widest disparity was observed between assessors with and those without critical care experience (66% versus 39%), as well as assessors who were teaching the course and those who were not (68% versus 49%). Qualitative analysis revealed three dominant themes within participants' transcripts. The themes focused on determining student safety, and the use of personal perceptions and clinical experience to determine competency.Findings indicate that assessors' individual perceptions and clinical experience have the potential to influence and determine undergraduate nursing students OSCA results.Development of criteria standards and objective assessment may be enhanced by greater involvement of assessors and thorough education and training within the context of student assessments.

A proposed model of cognitive approach to assess clinical performance of dental students

“…For example, an increased level of content knowledge of assessors affects their rating. Moreover, those assessors who have a direct interaction with students tend to give higher marks, which could be intentional, when compared to those assessors who present only during the assessment task without having previous interaction with students [16]. It has been reported and identified that content knowledge is one of the major factors behind a good assessment.…”

Untitled

“…[59] Previous studies performed in the UK and Australia report OSCE's lacks consistency and reliability. [59,60] This lack of uniformity translates into a significant variation of the scores among assessors. [60] The paucity of Canadian nursing education research on the validity and reliability of Objective Structured Clinical Examination adds to the current void of evidence on the impact of remote robots on teaching clinical skills.…”

“…[22] Nurse researchers report a wide variation in pass rates between assessors due to subjective biases. [59] East et al explain that subjective biases relate to instructors' perceptions of clinical competency. [59] Previous studies performed in the UK and Australia report OSCE's lacks consistency and reliability.…”

“…[59] East et al explain that subjective biases relate to instructors' perceptions of clinical competency. [59] Previous studies performed in the UK and Australia report OSCE's lacks consistency and reliability. [59,60] This lack of uniformity translates into a significant variation of the scores among assessors.…”

Screens or MP4s: Acquiring Clinical Competencies Through the Use of e-OSPEs in Occupational Therapy

“…Furthermore, the Standards Generating Body (which was a temporary working group consisting of representatives of eight different universities that developed qualification standards) (HPCSA, 2017) of the profession indicated that students should demonstrate (a) the ability to assess the occupational performance of individuals, groups, and communities effectively in order to determine occupational performance problems requiring intervention; (b) the use of appropriate assessment techniques and processes; and (c) the consideration of individual characteristics, cultures, and other unique needs and contexts of the individual, group, or community. One of the main methods of assessing practical skills in medical and health sciences training is the objective standardized clinical examination (OSCE) (Townsend, Mcllvenny, Miller, & Dunn, 2001), also referred to as the objective structured clinical assessment (OSCA) in nursing education (East, Peters, Halcomb, Raymond, & Salamonson, 2014). An OSPE provides a simulated environment in which to assess a student's competence in demonstrating clinical reasoning and the foundational skills needed for executing successful assessments and interventions.…”