Interrater Reliability of Standardized Actors Versus Nonactors in a Simulation Based Assessment of Interprofessional Collaboration

Dickter, David N.; Stielstra, Sorrel; Lineberry, Matthew

doi:10.1097/sih.0000000000000094

Cited by 16 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Assessment at the end of the IPE includes a realistic simulation for predoctoral students in the health care professions called the Ambulatory Team Observed Structured Clinical Evaluation (ATOSCE), an experience that provides a way for students to apply what they have learned in IPE to demonstrate their interprofessional competence and to be evaluated using a valid, reliable performance assessment 12, 13 . One of the coauthors participated in development of the simulation to ensure that the dental scope of practice was addressed.…”

Section: Methodsmentioning

confidence: 99%

“…In sample two, we also collected student performance results as measured by observers: two standardized patients who observed students and completed a performance checklist whose reliability was established in a prior study 13 . The observer ratings were mostly conducted live as the simulation happened, but the simulation was recorded as well, providing observers with an opportunity to review students' performance in further detail if necessary.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Dental Students' Perceived Value of Faculty vs. Peer Feedback on Non‐Technical Clinical Competency Assessments

Andrews

Dickter

Stielstra

et al. 2019

Journal of Dental Education

Self Cite

View full text Add to dashboard Cite

Although reviewing dental students' clinical competency assessments is an important aspect of instruction, finding time to give individual feedback to each student poses a challenge for faculty members, and some students may prefer to receive feedback from a peer. The aim of this study was to explore dental students' perceived value of feedback on their performance in a simulated patient care activity from either a faculty member or a peer. Participants were third‐ and fourth‐year dental students who had completed two years of interprofessional instruction and a videotaped objective structured clinical examination (OSCE) with standardized patients. Participants in two cohorts were randomly assigned to a faculty or peer feedback group. Cohort one (2015‐16) consisted of 66 students: 21 in faculty‐led groups, and 45 in peer‐led groups. Cohort two (2017) consisted of 60 students: 17 in faculty‐led groups, and 43 in peer‐led groups. In both types of pairings, the protocol consisted of jointly observing a video recording of student performance in the simulated patient encounter and discussing questions about the student's performance in non‐technical competencies such as communication, patient safety, scope of practice, and conflict resolution. For cohort two, prior to the feedback sessions, students in the peer feedback groups received a 60‐minute training on providing constructive feedback. All 126 students in the two cohorts completed an evaluation questionnaire after the experience. The results showed that students in both types of feedback sessions perceived value in the feedback and believed it enhanced their skills. However, students rated faculty feedback significantly higher (p<0.05) than peer feedback on nearly all dimensions. Perceived value did not differ by age, gender, class year, or OSCE performance. These results provide support for the value of peer feedback on nontechnical clinical competency assessments, though not as a substitute for faculty feedback.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Comparison of Dental Students' Perceived Value of Faculty vs. Peer Feedback on Non‐Technical Clinical Competency Assessments

Andrews

Dickter

Stielstra

et al. 2019

Journal of Dental Education

Self Cite

View full text Add to dashboard Cite

show abstract

“…14 The use of standardized scoring rubrics has been shown to produce a high level of interrater reliability. 15 This is further enhanced by the ability to record stations for asynchronous review, rather than requiring raters to rely solely on their memory of the encounter.…”

Section: The Ccacc Clinical Performance Examinationmentioning

confidence: 99%

A Call to Action From the California Consortium for the Assessment of Clinical Competence: Making the Case for Regional Collaboration

Nevins¹,

Boscardin²,

Kahn³

et al. 2022

Academic Medicine

View full text Add to dashboard Cite

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills Examination emphasizes the need for other reliable standardized assessments of medical student clinical skills. For 30 years, the California Consortium for the Assessment of Clinical Competence (CCACC) has collaborated in the development of clinical skills assessments and has become a valuable resource for clinicians, standardized patient educators, psychometricians, and medical educators. There are many merits to strong multi-institutional partnerships, including the integration of data across multiple schools to provide feedback to both students and curricula, pooled test performance statistics for analysis and quality assurance, shared best practices and resources, individual professional development, and opportunities for research and scholarship. The infrastructure of the CCACC allows member schools to adapt to a changing medical landscape, from emerging trends in clinical medicine to the limitations imposed by a global pandemic. In the absence of a national examination, there is now a greater need for all medical schools to develop a comprehensive, dynamic, and psychometrically sound assessment that accurately evaluates clinical skills. Medical schools working together in regional consortia have the opportunity to create and implement innovative and robust assessments that evaluate a wide range of clinical skills, ensure that medical students have met an expected level of clinical competency before graduation, and provide a framework that contributes to ongoing efforts for the development of new national clinical skills standards.An Academic Medicine Podcast episode featuring this article is available wherever you get your podcasts.

show abstract

“…Like previous studies using SPs, our results indicate that SPs may be a critical resource for evaluating and strengthening students' communication skills. Evidence from several reports suggest that SPs assess target clinical and communication behaviors with strong inter-rater reliability (Shirazi et al, 2014), and may be even more reliable than extensively trained non-SP observers (Dickter et al, 2015). SP input is also unaffected by potential biases of faculty whose assessments may be influenced by their previous knowledge and perceptions of the student participants (Brinkman et al, 2006).…”

Section: Clinical and Educational Implicationsmentioning

confidence: 99%

Original Interprofessional Simulations to Train Students in CSD and Related Health Professions in Team-Based Health Care

Eichorn

Zarn

Moncrieff

et al. 2020

Communication Disorders Quarterly

View full text Add to dashboard Cite

Students in health professions receive extensive training across the scope of practice within their profession; however, most lack experience in performing clinical functions as a member of a coordinated health care team. We describe outcomes of a quasi-experimental study that used simulations to target interprofessional practice competencies. Simulations incorporated clinical knowledge from five professions, and used standardized patients to provide students with a low-stakes opportunity to apply knowledge and skills. Students observed one simulation and participated in another. Post-simulation quantitative and qualitative data were collected by having students, faculty, and standardized patients rate collaboration and decision-making skills for each interprofessional team. Results indicate differences in ratings based on simulation round and rater group. Findings highlight the need for focused training in the use of clear and empathic communication by interprofessional teams. Results also demonstrate the value of feedback from standardized patients who can provide important perspectives regarding the quality of clinical interactions.

show abstract

Interrater Reliability of Standardized Actors Versus Nonactors in a Simulation Based Assessment of Interprofessional Collaboration

Cited by 16 publications

References 16 publications

Comparison of Dental Students' Perceived Value of Faculty vs. Peer Feedback on Non‐Technical Clinical Competency Assessments

Comparison of Dental Students' Perceived Value of Faculty vs. Peer Feedback on Non‐Technical Clinical Competency Assessments

A Call to Action From the California Consortium for the Assessment of Clinical Competence: Making the Case for Regional Collaboration

Original Interprofessional Simulations to Train Students in CSD and Related Health Professions in Team-Based Health Care

Contact Info

Product

Resources

About