Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems

Abstract

Many technical approaches have been proposed for ensuring that decisions made by machine learning systems are fair, but few of these proposals have been stress-tested in real-world systems. This paper presents an example of one team's approach to the challenge of applying algorithmic fairness approaches to complex production systems within the context of a large technology company. We discuss how we disentangle normative questions of product and policy design (like, "how should the system trade off between different stakeholders' interests and needs?") from empirical questions of system implementation (like, "is the system achieving the desired tradeoff in practice?"). We also present an approach for answering questions of the latter sort, which allows us to measure how machine learning systems and human labelers are making these tradeoffs across different relevant groups. We hope our experience integrating fairness tools and approaches into large-scale and complex production systems will be useful to other practitioners facing similar challenges, and illuminating to academics and researchers looking to better address the needs of practitioners.

Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare

“…A key consequence of the analysis presented thus far is that, subject to the assumptions detailed in section 2.3, the optimal threshold rule applied to a predictive model that outputs a continuous-valued risk score is based directly on the calibration characteristics of the model and the assumed expected costs or utilities of classification errors that encapsulate the effectiveness of the intervention and the preferences for downstream benefits and harms. As has been argued in related work [15,18,31,50], it follows that if the model is calibrated for each subgroup, the decision threshold that maximizes expected utility and net benefit for each subgroup is the same when the expected utilities associated with each classification error do not change across subgroups. We verify this claim in simulation in supplementary section A1 (Supplementary Figure A1).…”

“…To define the full MMD, each of the three expectations in equation ( 30) are replaced with weighted variants analogous to equation (31). Now, we consider the operationalization of equation ( 9) to penalize differences in model performance metrics.…”

“…We verify this claim in simulation in supplementary section A1 (Supplementary Figure A1). Furthermore, in this case, sufficiency implies that the use of a consistent threshold on the risk score for all subgroups corresponds to the use of a consistent threshold on the conditional utility U cond across subgroups, corresponding to an intuitive notion of fairness even in the case that the chosen decision threshold is not necessarily optimal [18,31]. However, we note that this can still be a misleading notion of fairness given that it does not account for heterogeneity in the outcome not accounted for by the model under consideration [18].…”

Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support

“…It is therefore difficult to, as some have claimed, "disentangle normative questions of product design... from empirical questions of system implementation" [5], given the ways in which practitioners' work practices are shaped by the organizational contexts in which they are embedded [e.g., 44,47,63].…”

“…In the interest of reflexivity [16,43], we acknowledge that our perspectives and approaches to research are shaped by our own experiences and positionality. Specifically, we are researchers living and working in the U.S., primarily working in industry 5 with years of experience working closely with AI practitioners on projects related to the fairness of AI systems. In addition, we come from a mix of disciplinary backgrounds, including AI and HCI, which we have drawn on to conduct prior research into sociotechnical approaches to identifying, assessing, and mitigating fairness-related harms caused by AI systems.…”