The role of simulation in high-stakes assessment

Dupré, Jonathan; Naik, Viren N.

doi:10.1016/j.bjae.2020.12.002

Cited by 11 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While SBA has been used for highstakes assessment, [24][25][26] little research has explored how the same raters think about their entrustment decisions across WBA and SBA. Key questions include what influence these assessment settings may have on raters' cognitive processes when assessing EPAs and how this influence affects how data from WBAs and SBAs can be used to make decisions in CBME programs.…”

Section: Discussionmentioning

confidence: 99%

Variable or Fixed? Exploring Entrustment Decision Making in Workplace- and Simulation-Based Assessments

et al. 2022

View full text Add to dashboard Cite

Purpose Many models of competency-based medical education (CBME) emphasize assessing entrustable professional activities (EPAs). Despite the centrality of EPAs, researchers have not compared rater entrustment decisions for the same EPA across workplace- and simulation-based assessments. This study aimed to explore rater entrustment decision making across these 2 assessment settings. Method An interview-based study using a constructivist grounded theory approach was conducted. Gastroenterology faculty at the University of Toronto and the University of Calgary completed EPA assessments of trainees’ endoscopic polypectomy performance in both workplace and simulation settings between November 2019 and January 2021. After each assessment, raters were interviewed to explore how and why they made entrustment decisions within and across settings. Transcribed interview data were coded iteratively using constant comparison to generate themes. Results Analysis of 20 interviews with 10 raters found that participants (1) held multiple meanings of entrustment and expressed variability in how they justified their entrustment decisions and scoring, (2) held personal caveats for making entrustment decisions “comfortably” (i.e., authenticity, task-related variability, opportunity to assess trainee responses to adverse events, and the opportunity to observe multiple performances over time), (3) experienced cognitive tensions between formative and summative purposes when assessing EPAs, and (4) experienced relative freedom when using simulation to formatively assess EPAs but constraint when using only simulation-based assessments for entrustment decision making. Conclusions Participants spoke about and defined entrustment variably, which appeared to produce variability in how they judged entrustment across participants and within and across assessment settings. These rater idiosyncrasies suggest that programs implementing CBME must consider how such variability affects the aggregation of EPA assessments, especially those collected in different settings. Program leaders might also consider how to fulfill raters’ criteria for comfortably making entrustment decisions by ensuring clear definitions and purposes when designing and integrating workplace- and simulation-based assessments.

show abstract

Section: Discussionmentioning

confidence: 99%

Variable or Fixed? Exploring Entrustment Decision Making in Workplace- and Simulation-Based Assessments

et al. 2022

View full text Add to dashboard Cite

show abstract

“…37 Simulation can also be beneficial when certain clinical events are critical but rare, and when it is not appropriate or safe to delegate responsibilities to a trainee solely for the purpose of undertaking an assessment. 38 In anesthesiology and emergency medicine training, in which patient care is also acute and dynamic, simulation-based milestone assessments have been shown to be valid measures of resident competency and correlate with clinical evaluations. [39][40][41] Relatedly, the performance on a regular objective structured clinical examination (OSCE)-based milestone assessment can also provide evidence of longitudinal growth in competence in anesthesiology residents.…”

Section: What Work Needs To Be Done For Cbme?mentioning

confidence: 99%

Competency-Based Medical Training in Anesthesiology: Has It Delivered on the Promise of Better Education?

Kealey

Naik

2022

Anesthesia &Amp; Analgesia

View full text Add to dashboard Cite

“…A simulation-based staged learning process that goes from “knows” and “knows how” to “shows how” and “does” might allow residents the practice of crisis management without any risk to patient safety. [ 9 10 ] At a minimum, simulation for basic and advanced life support, ultrasound, central vein cannulations, and difficult airway mannequins should be considered and used for the training of the residents.…”

Section: Incorporating Changes In the Pg Curriculummentioning

confidence: 99%