An Investigation of the Standardised Patient Interview Rating Scale (SPIRS) for the Assessment of Speech Pathology Students in a Simulation Clinic

Abstract

Standardised patients (SPs) are increasingly utilised in health sciences education to assist students in the development of clinical competencies, including interviewing skills. This study investigated the development and validation of a rating scale for formative assessment of speech pathology students in an interview with an SP. Participants in this study were 76 undergraduate speech pathology students and 10 clinical educators who participated in a simulated clinic module. As part of the module, pairs of students interviewed an SP portraying a parent of a child with speech delay. The Standardised Patient Interview Rating Scale (SPIRS) was developed to assess students' foundation clinical competencies of communication, interviewing and professional practice skills. Students' interviews were videotaped, rated individually on the SPIRS by the clinical educator, and later re-rated by an expert rater. Data were analysed to determine the content validity, internal consistency, and inter-rater reliability of the tool. In addition, descriptive statistics were used to report student performance levels. Results indicated that the SPIRS had good content validity and internal consistency but that there may be some redundancy in individual items. An acceptable level of inter-rater reliability was achieved. Students generally scored highly, with non-verbal communication being the easiest and professional practice the most difficult skill to demonstrate. The SPIRS was found to be an appropriate tool for formative assessment of students in this simulated clinic module. Recommendations for improving its reliability were made. Further research is required to investigate use of the SPIRS as an assessment tool in other contexts utilising standardised patients.

An Investigation of Agreement between Clinical Educators and Peers in Rating Speech Pathology Students’ Interview Skills

“…The finding that PEA between peers and clinical educators was higher for SPIRS item 1.1 (nonverbal communication) has similarities to the findings of Hill, Davidson, and Theodoros (2015) who reported the highest level of agreement in SPIRS item 1.1. Rating of non-verbal communication may have had higher agreement as eye-contact, gesture and body language are observable, do not require interpretation and can be objectively assessed (Hill, Davidson, and Theodoros 2015). In contrast, agreement between peers and clinical educators was low on SPIRS item 2 (professional practice) in both weeks 4 and 8.…”

“…The findings of the current study are of interest when compared with those of Hill, Davidson, and Theodoros (2015), who found acceptable levels of rater agreement between an 'expert rater' (a rater with extensive experience in student competency rating) and 10 other clinical educators (mean PEA of 82.06%). The level of expertise and experience of raters may have contributed to the discrepancy between the Hill, Davidson, and Theodoros (2015) study and the current study.…”

“…One way to increase the knowledge and experience of raters is to provide additional training. Hill, Davidson, and Theodoros (2015) reported that the clinical educators participating in their study received group training in use of the SPIRS with practical application. Students in the current study received an introduction to the SPIRS and brief training only.…”

“…In addition, an overall rating of interview performance is provided (also a 5point scale). It is a validated tool with high internal consistency (α = 0.743) and acceptable levels of inter-rater reliability (Hill, Davidson, and Theodoros 2015). This tool was ideal for use in the current study as it had been validated with clinical educators undertaking assessment of students in the same clinical context.…”

A framework to support the development of quality simulation‐based learning programmes in speech–language pathology

“…, Wallace ), or a published tool such as the Standardised Patient Interview Rating Scale used in SLP (SPIRS) (Hill et al . ). Similar to the processes and considerations described in Stage 1—Leadership, Consultation and Evidence and Stage 2—National Forum, the iterative process and interaction between the context, learner and assessment requirements is important to establish before development of the learning objectives.…”