“My Assessments Are Biased!” Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education

Abstract

Assessing learners is foundational to their training and developmental growth throughout the medical education continuum. However, growing evidence shows the prevalence and impact of harmful bias in assessments in medical education, accelerating the urgency to identify solutions. Assessment bias presents a critical problem for all stages of learning and the broader educational system. Bias poses significant challenges to learners, disrupts the learning environment, and threatens the pathway and transition of learners into health professionals. While the topic of assessment bias has been examined within the context of measurement literature, limited guidance and solutions exist for learners in medical education, particularly in the clinical environment. This article presents an overview of assessment bias, focusing on clinical learners. A definition of bias and its manifestations in assessments are presented. Consequences of assessment bias are discussed within the contexts of validity and fairness and their impact on learners, patients/caregivers, and the broader field of medicine. Messick’s unified validity framework is used to contextualize assessment bias; in addition, perspectives from sociocultural contexts are incorporated into the discussion to elaborate the nuanced implications in the clinical training environment. Discussions of these topics are conceptualized within the literature and the interventions used to date. The article concludes with practical recommendations to overcome bias and to develop an ideal assessment system. Recommendations address articulating values to guide assessment, designing assessment to foster learning and outcomes, attending to assessment procedures, promoting continuous quality improvement of assessment, and fostering equitable learning and assessment environments.

Josiah Macy Jr. Foundation Conference on Ensuring Fairness in Medical Education Assessment: Conference Recommendations Report

“…Harmful bias, which unfairly and negatively impacts those toward whom it is directed, exists in all forms in medical education and permeates institutional cultures , climates , and learning and working environments. In terms of assessment, “bias that arises from assessors’ categorizations, interpretations, and assumptions that are not based on [unbiased assessment of] learners’ ability and performance disproportionately and negatively affects learners [from groups marginalized in medicine].” 4…”

“…Hauer and colleagues 4 explain that negative consequences of bias affect many different marginalized groups of learners in medicine, including non-White racial/ethnic groups (often referred to as underrepresented in medicine or URIM, although not all learners of color are underrepresented in medicine); women; learners with disabilities; learners who are lesbian, gay, bisexual, transgender, or queer; and international medical graduates. Given the range of marginalized groups identified in the literature on bias in assessment, it stands to reason that learners in every marginalized group have, or are at risk for, problematic assessment experiences related to bias.…”

“…In “‘My Assessments Are Biased!’ Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education,” 4 the authors examine bias in learner assessment from both a measurement perspective and a sociocultural perspective. In terms of measurement, harmful bias can be introduced during assessment design and implementation—perhaps through test questions written in such a way that one group of learners (e.g., men) is unfairly advantaged while another (e.g., women) is unfairly disadvantaged—causing the results to inaccurately reflect learner performance.…”

“…Mounting evidence suggests that “bias plagues assessment” and can impede learners’ developmental progress and prevent some from advancing in the medical profession. 4…”

The Next Era of Assessment: Can Ensuring High-Quality, Equitable Patient Care Be the Defining Characteristic?

“…Education scholars who attempt to redefine assessment for the purpose of advancing health equity should explicitly seek to minimize the health equity tourism discussed previously through intentional, cross-disciplinary collaborations. 44,87,88 Medical educators must not only involve health equity scholars as core team members in this process of redefining the objectives of assessment but also build trusting collaborations with the communities that they serve to help them redefine the standards of assessment. A recent Josiah Macy Jr. Foundation conference focused on ensuring fairness and equity in assessment offered several recommendations: creating a culture, climate, and learning and working environment that promotes, values, and demonstrates fairness in assessment; executing assessment systems that promote fairness and equity; equipping assessors and others in the systems to ensure assessment systems that eliminate harmful bias and foster trusting relationships; using assessment data for the purpose it was designed for; and requiring all members of the medical education community commit to and work toward achieving fairness and equity in assessment.…”