Gender Bias in Simulation-Based Assessments of Emergency Medicine Residents

Siegelman, Jeffrey; Lall, Michelle D.; Lee, Lindsay; Moran, Tim P.; Wallenstein, Joshua; Shah, Bijal

doi:10.4300/jgme-d-18-00059.1

Cited by 14 publications

(23 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Resultsmentioning

confidence: 99%

“…Twenty studies were conducted in the United States, 14–33 13 were performed in Canada, 34–46 three took place in Australia, 47–49 and two were conducted in Taiwan 50,51 . Thirty‐three studies involved resident physicians, 15,17–23,25–32,34–47,49,50 four studies involved medical students, 14,16,24,33 and one did not report the learner level 48 . Twenty‐two were prospective studies, 14,16,17,22–28,30,31,33,37,38,40–42,46–48,51 nine were retrospective, 18–20,32,35,39,44,45,50 five were qualitative studies, 21,29,34,43,49 and two were descriptive studies 15,36 .…”

Section: Resultsmentioning

confidence: 99%

“…Twenty studies were conducted in the United States, [14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33] 13 were performed in Canada, [34][35][36][37][38][39][40][41][42][43][44][45][46] three took place in Australia, [47][48][49] and two were conducted in Taiwan. 50,51 Thirty-three studies involved resident 1.…”

Section: Summary Of Findingsmentioning

confidence: 99%

“…Develop standardized training programs and assessments for procedural skill acquisition (such as those for central line insertion), starting with no-risk methods such as simulated, cadaveric, or OSCE experiences and concluding with direct observation assessment during actual patient care and correlation to complications and patient outcomes. AEM EDUCATION AND TRAINING • www.aem-e-t.com physicians, 15,[17][18][19][20][21][22][23][25][26][27][28][29][30][31][32][34][35][36][37][38][39][40][41][42][43][44][45][46][47]49,50 four studies involved medical students, 14,16,24,33 and one did not report the learner level. 48 Twenty-two were prospective studies, 14,16,17,[22][23][24][25][26][27][28]…”

Section: Summary Of Findingsmentioning

confidence: 99%

See 3 more Smart Citations

Direct Observation Tools in Emergency Medicine: A Systematic Review of the Literature

Gottlieb¹,

Jordan²,

Siegelman³

et al. 2020

AEM Education and Training

Self Cite

View full text Add to dashboard Cite

Objectives: Direct observation is important for assessing the competency of medical learners. Multiple tools have been described in other fields, although the degree of emergency medicine-specific literature is unclear. This review sought to summarize the current literature on direct observation tools in the emergency department (ED) setting. Methods:We searched PubMed, Scopus, CINAHL, the Cochrane Central Register of Clinical Trials, the Cochrane Database of Systematic Reviews, ERIC, PsycINFO, and Google Scholar from 2012 to 2020 for publications on direct observation tools in the ED setting. Data were dual extracted into a predefined worksheet, and quality analysis was performed using the Medical Education Research Study Quality Instrument.Results: We identified 38 publications, comprising 2,977 learners. Fifteen different tools were described. The most commonly assessed tools included the Milestones (nine studies), Observed Structured Clinical Exercises (seven studies), the McMaster Modular Assessment Program (six studies), Queen's Simulation Assessment Test (five studies), and the mini-Clinical Evaluation Exercise (four studies). Most of the studies were performed in a single institution, and there were limited validity or reliability assessments reported. Conclusions:The number of publications on direct observation tools for the ED setting has markedly increased.However, there remains a need for stronger internal and external validity data.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Summary Of Findingsmentioning

confidence: 99%

Section: Summary Of Findingsmentioning

confidence: 99%

See 2 more Smart Citations

Direct Observation Tools in Emergency Medicine: A Systematic Review of the Literature

Gottlieb¹,

Jordan²,

Siegelman³

et al. 2020

AEM Education and Training

Self Cite

View full text Add to dashboard Cite

show abstract

“…Utilizing standardized assessments in controlled or simulated environments may also be helpful, especially if there are discrepant evaluations for trainees. In a 2018 study of medical simulations for EM residents conducted with a scripted and standardized assessments found that there were no differences in scores due to resident or rater gender, supporting these assessments to limit bias 17 . These simulated assessments can be part of a larger portfolio of multisource evaluation tools utilized by CCCs to consider when assigning semiannual competency ratings.…”

Section: Discussionmentioning

confidence: 93%

Gender Differences Among Milestone Assessments in a National Sample of Pediatric Emergency Medicine Fellowship Programs

Zuckerbraun

Levasseur

Kou

et al. 2020

AEM Education and Training

View full text Add to dashboard Cite

Background: Understanding gender gaps in trainee evaluations is critical because these may ultimately determine the duration of training. Currently, no studies describe the influence of gender on the evaluation of pediatric emergency medicine (PEM) fellows. Objective:The objective of our study was to compare milestone scores of female versus male PEM fellows. Methods: This is a multicenter retrospective cohort study of a national sample of PEM fellows from July 2014 to June 2018. Accreditation Council for Medical Education (ACGME) subcompetencies are scored on a 5-point scale and span six domains: patient care (PC), medical knowledge, systems-based practice, practice-based learning and improvement, professionalism, and interpersonal and communication skills (ICS). Summative assessments of the 23 PEM subcompetencies are assigned by each program's clinical competency committee and submitted semiannually for each fellow. Program directors voluntarily provided deidentified ACGME milestone reports. Demographics including sex, program region, and type of residency were collected. Descriptive analysis of milestones was performed for each year of fellowship. Multivariate analyses evaluated the difference in scores by sex for each of the subcompetencies.Results: Forty-eight geographically diverse programs participated, yielding data for 639 fellows (66% of all PEM fellows nationally); sex was recorded for 604 fellows, of whom 67% were female. When comparing the mean milestone scores in each of the six domains, there were no differences by sex in any year of training. When comparing scores within each of the 23 subcompetencies and correcting the significance level for comparison of multiple milestones, the scores for PC3 and ICS2 were significantly, albeit not meaningfully, higher for females.

show abstract

Assessment of Implicit Gender Bias During Evaluation of Procedural Competency Among Emergency Medicine Residents

et al. 2022

View full text Add to dashboard Cite

IMPORTANCE Gender disparities exist throughout medicine. Recent studies have highlighted an attainment gap between male and female residents in performance evaluations on Accreditation Council for Graduate Medical Education (ACGME) milestones. Because of difficulties in blinding evaluators to gender, it remains unclear whether these observed disparities are because of implicit bias or other causes. OBJECTIVE To estimate the magnitude of implicit gender bias in assessments of procedural competency in emergency medicine residents and whether the gender of the evaluator is associated with identified implicit gender bias. DESIGN, SETTING, AND PARTICIPANTSA cross-sectional study was performed from 2018 to 2020 in which emergency medicine residency faculty assessed procedural competency by evaluating videos of residents performing 3 procedures in a simulated environment. They were blinded to the intent of the study. Proceduralists were filmed performing each procedure from 2 different viewpoints simultaneously by 2 different cameras. One was a gender-blinded (ie, hands-only) view, and the other a wide-angled gender-evident (ie, whole-body) view. The faculty evaluators viewed videos in a random order and assessed procedural competency on a global rating scale with extensive validity evidence for the evaluation of video-recorded procedural performance. MAIN OUTCOMES AND MEASURESThe primary outcome was to determine if there was a difference in the evaluation of procedural competency based on gender. The secondary outcome was to determine if there was a difference in the evaluations based on the gender of the evaluator. RESULTSFifty-one faculty evaluators enrolled from 19 states, with 22 male participants (43.1%), 29 female participants (56.9%), and a mean (SD) age of 37 (6.4) years. Each evaluator assessed all 60 procedures: 30 gender-blinded (hands-only view) videos and 30 identical gender-evident (wide angle) videos. There were no statistically significant differences in the study evaluators' scores of the proceduralists based on their gender, and the gender of the evaluator was not associated with the difference in mean scores. CONCLUSIONS AND RELEVANCEIn this study, we did not identify a difference in the evaluation of procedural competency based upon the gender of the resident proceduralist or the gender of the faculty evaluator.

show abstract

Gender Bias in Simulation-Based Assessments of Emergency Medicine Residents

Abstract: Background Gender-related disparities persist in medicine and medical education. Prior work has found differences in medical education assessments based on gender.

Cited by 14 publications

References 18 publications

Direct Observation Tools in Emergency Medicine: A Systematic Review of the Literature

Direct Observation Tools in Emergency Medicine: A Systematic Review of the Literature

Gender Differences Among Milestone Assessments in a National Sample of Pediatric Emergency Medicine Fellowship Programs

Assessment of Implicit Gender Bias During Evaluation of Procedural Competency Among Emergency Medicine Residents

Contact Info

Product

Resources

About