Viability of Virtual Skills-based Assessments Focused on Communication

Abstract

Objective. To evaluate faculty and student perceptions and performance of virtual compared to in-person skills-based assessments focused on communication. Methods. Providing sufficient formative and summative feedback is a challenge, particularly in the context of skills-based assessments. In spring 2020, two 12-item questionnaires, one each for students and faculty, were designed to assess perceptions of virtual skills-based assessments. The survey was distributed via QuestionPro© to second and third professional year pharmacy students and faculty who participated in a virtual skills-based assessment. Scores from the spring 2020 virtual skills-based assessment were compared to the in-person skills-based assessment that took place in spring 2019. Results. Of the 19 faculty and 279 students invited to participate, 18 (94.7%) faculty and 241 (86.4%) students responded. The majority of faculty (88.9%) and students (63.5%) perceived the virtual skills-based assessments to be effective at simulating an interaction. However, only 33.3% of faculty and 28.6% of students preferred the virtual environment. There was not a significant difference in the percent of third year students scoring 80% or higher between in-person and virtual assessments for patient consultation and subjective, objective, assessment, and plan (SOAP) note skills. Conclusion. Students and faculty reported the virtual assessment provided an opportunity for an appropriate assessment of student communication skills. However, despite feelings that the assessment was appropriate, a strong preference for future virtual skills-based assessments was not observed.

Remote OSCE Experience: What First Year Pharmacy Students Liked, Learned, and Suggested for Future Implementations

“…However, students in this study agreed that there were differences in "difficulty, stress, interpreting patient cues, and in maintaining eye contact and tone of voice between the two means of consultation" [7]. Recently, VanLangen and colleagues found virtual skills-based assessments to be an appropriate mechanism to assess student communication skills; however, they did not observe a strong preference for virtual skills-based assessments in the future [8]. Similar research from nursing and medicine describes telehealth OSCE simulation yet fails to explore student perspectives [9,10].…”

Assessing Pharmacy Student Performance and Perceptions on Counseling Skills Through a Simulated Telehealth Encounter

“…25 Recently, patient counseling and communication skills were assessed via virtual platforms whereby students and faculty considered the virtual assessment appropriate in assessing student communication skills, however, a strong preference for future virtual skills-based assessments was not observed. 35 In our study, despite reported anxiety with a new assessment, students were very satisfied and looked forward to undergoing similar assessments in the future. We think that the rigorous process adopted in developing the case scenarios and associated evaluation rubrics, in addition to student orientation and session piloting were instrumental in assuring student familiarity with the new assessment and the level of performance expected from them.…”

“…33 Skill-based assessments have also been conducted virtually, including formative and summative OSCE; students were found to perform comparably to those conducted in-person. [34][35] Several studies have been conducted to compare study behaviors and grades of students in on and off campuses licensure examinations, and have found no significant differences in learning outcomes achievement between the two. [36][37][38] The program at Ajman University, United Arab Emirates (UAE) offers a four-year, 160-credit hour Bachelor of Pharmacy degree.…”

Evaluation of a Rubric to Assess Pharmacy Student Performance in a Telehealth Simulation Exercise

“…[8][9][10][11][12] Student performance scores on virtual skills-based assessments compared to in-person have also been published, but a method for evaluating competency of skills performed in a telehealth activity that can be used among different evaluator types is not currently available. [12][13][14] Telehealth etiquette refers to unique behaviors outside of typical professional actions that are used to conduct an effective telehealth visit. 15 Both verbal and non-verbal communication skills are components of telehealth etiquette that lead to increased patient satisfaction and successful interactions.…”