Fairness Without Demographic Data: A Survey of Approaches

Ashurst, Carolyn; Weller, Adrian

doi:10.1145/3617694.3623234

Cited by 7 publications

(3 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2. Excluding data related to race reduces the share of low-income applicants in the top pool by a small but statistically significant amount as compared to the ML baseline (from 31% to 26%, 𝑝 < 0.001), but it does not significantly change the share of first-gen applicants (from 27% to 26%, 𝑝 = 0.04 14 ). As with URM status, excluding all uncontrollable features further exacerbates the reduction in low-income (16%, 𝑝 < 0.001) and first-gen (11%, 𝑝 < 0.001) applicants in the top pool, but excluding applicants' intended major does not impact socioeconomic diversity as compared to the ML baseline (the share of low-income applicants increases to 32% and the share of first-generation applicants increases to 29%, but neither change is statistically significant (𝑝 = 0.12 and 𝑝 = 0.01, 15 respectively)).…”

Section: Groupmentioning

confidence: 91%

“…In addition, other than the naive baseline (𝑝 < 0.001), which sorts students by highest math instruction and test scores, no model was statistically significantly better or worse than the ML baseline at identifying students who were 14 Not significant after accounting for multiple comparisons 15 Not significant after accounting for multiple comparisons actually admitted or waitlisted by the case institution. About half of the students included in the top pool are actually admitted or waitlisted across the ML baseline (47%), No race (49%), No major (46%), and No uncontrollable features (46%) models.…”

Section: Groupmentioning

confidence: 95%

“…It will therefore be important to identify alternative ranking approaches that can mitigate this impact, or that can increase other forms of diversity, like the share of top-ranked applicants that are first-generation or low-income (FGLI) students. Prior work related to equity and access in algorithms, including Thomas et al [65]'s framework for positive action, Arif Khan et al [12]'s decision procedures for substantive equality of opportunity, and Borgs et al [17]'s approach to algorithmic greenlining, is likely to be instructive here; as will prior work on measuring and improving fairness without access to demographic data [14]. We also show that arbitrariness can have a major impact on how applicants are ranked by ML models.…”

Section: Opportunities For Future Workmentioning

confidence: 99%

See 2 more Smart Citations

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

Lee,

Harvey,

Zhou

et al. 2024

Preprint

View full text Add to dashboard Cite

Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-2024 admissions cycle, these colleges faced unprecedented challenges to doing so. First, the number of applications has been steadily growing year-over-year. Second, test-optional policies that have remained in place since the COVID-19 pandemic limit access to key information that has historically been predictive of academic success. Most recently, longstanding debates over affirmative action culminated in the Supreme Court banning race-conscious admissions. Colleges have explored machine learning (ML) models to address the issues of scale and missing test scores, often via ranking algorithms intended to allow human reviewers to focus attention on ‘top’ applicants. However, the Court’s ruling will force changes to these models, which were previously able to consider race as a factor in ranking. There is currently a poor understanding of how these mandated changes will shape applicant ranking algorithms, and, by extension, admitted classes. We seek to address this by quantifying the impact of different admission policies on the applications prioritized for review. We show that removing race data from a previously developed applicant ranking algorithm reduces the diversity of the top-ranked pool of applicants without meaningfully increasing the academic merit of that pool. We contextualize this impact by showing that excluding data on applicant race has a greater impact than excluding other potentially informative variables like intended majors. Finally, we measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness (i.e., how much an applicant’s rank changes across models that use the same policy but are trained on bootstrapped samples from the same dataset). We find that any given policy has a high degree of arbitrariness (i.e. at most 9% of applicants are consistently ranked in the top 20%), and that removing race data from the ranking algorithm increases arbitrariness in outcomes for most applicants.

show abstract

Section: Groupmentioning

confidence: 91%

Section: Groupmentioning

confidence: 95%

Section: Opportunities For Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

Lee,

Harvey,

Zhou

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Using Backbone Foundation Model for Evaluating Fairness in Chest Radiography Without Demographic Data

Queiroz,

Anjos,

Berton

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Policy advice and best practices on bias and fairness in AI

Alvarez,

Colmenarejo,

Elobaid

et al. 2024

Ethics Inf Technol

View full text Add to dashboard Cite

The literature addressing bias and fairness in AI models (fair-AI) is growing at a fast pace, making it difficult for novel researchers and practitioners to have a bird’s-eye view picture of the field. In particular, many policy initiatives, standards, and best practices in fair-AI have been proposed for setting principles, procedures, and knowledge bases to guide and operationalize the management of bias and fairness. The first objective of this paper is to concisely survey the state-of-the-art of fair-AI methods and resources, and the main policies on bias in AI, with the aim of providing such a bird’s-eye guidance for both researchers and practitioners. The second objective of the paper is to contribute to the policy advice and best practices state-of-the-art by leveraging from the results of the NoBIAS research project. We present and discuss a few relevant topics organized around the NoBIAS architecture, which is made up of a Legal Layer, focusing on the European Union context, and a Bias Management Layer, focusing on understanding, mitigating, and accounting for bias.

show abstract

Fairness Without Demographic Data: A Survey of Approaches

Cited by 7 publications

References 53 publications

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

Using Backbone Foundation Model for Evaluating Fairness in Chest Radiography Without Demographic Data

Policy advice and best practices on bias and fairness in AI

Contact Info

Product

Resources

About