Counterfactual risk assessments, evaluation, and fairness

Abstract

Algorithmic risk assessments are increasingly used to help humans make decisions in high-stakes settings, such as medicine, criminal justice and education. In each of these cases, the purpose of the risk assessment tool is to inform actions, such as medical treatments or release conditions, often with the aim of reducing the likelihood of an adverse event such as hospital readmission or recidivism. Problematically, most tools are trained and evaluated on historical data in which the outcomes observed depend on the historical decision-making policy. These tools thus reflect risk under the historical policy, rather than under the different decision options that the tool is intended to inform. Even when tools are constructed to predict risk under a specific decision, they are often improperly evaluated as predictors of the target outcome. Focusing on the evaluation task, in this paper we define counterfactual analogues of common predictive performance and algorithmic fairness metrics that we argue are better suited for the decision-making context. We introduce a new method for estimating the proposed metrics using doubly robust estimation. We provide theoretical results that show that only under strong conditions can fairness according to the standard metric and the counterfactual metric simultaneously hold. Consequently, fairness-promoting methods that target parity in a standard fairness metric may-and as we show empirically, do-induce greater imbalance in the counterfactual analogue. We provide empirical comparisons on both synthetic data and a real world child welfare dataset to demonstrate how the proposed method improves upon standard practice.

Who is analysing what? The opportunities, risks and implications of using predictive risk modelling with Indigenous Australians in child protection: A scoping review

“…With regard to methodological approaches, four studies presented the results from a range of different models (Chouldechova et al, 2018; Rea & Erasmus, 2017; Vaithianathan et al, 2017; Wilson et al, 2015). In the end, four studies used regression‐based models as final models (Chouldechova et al, 2018; Rea & Erasmus, 2017; Vaithianathan et al, 2013, 2017; Wilson et al, 2015), three studies based their predictions on random forests (Chouldechova et al, 2018; Coston et al, 2020; Wilson et al, 2015), and one study did not specify the actual model used (Oregon Department of Human Service, 2019).…”

“…Three US‐based studies focusing on the Allegheny County project used a linked dataset from a child maltreatment hotline and other administrative data sources. However, Chouldechova et al (2018) used OOHC placement as the dependent variable for their model, while Coston et al (2020) used re‐referral within 6 months as the outcome for their risk predictions. The latter study had a slightly different focus as its aim was a comparison of counterfactual risk assessment with observational PRMs.…”

“…Besides these challenges, predictive models in child protection settings may also be difficult to validate, since predictions are often concerned with outcomes such as OOHC placements, re‐reporting or re‐substantiation. In fact, the accuracy of the predictions is often untestable since child protection is, by its very own purpose, an intervention‐based system, creating challenges for the validation of the predictive ability of models (Chouldechova et al, 2018; Coston et al, 2020; Munro, 2019).…”

“…Given the limitations of PRMs and the potentially significant costs of inaccuracies in predictions to some of the most vulnerable members of our society, a growing body of research has concerned itself with addressing biases of PRMs and the fairness of machine learning algorithms in social settings (Chouldechova & G’Sell, 2017; Keddell, 2019; Mehrabi et al, 2019). In this context, fairness is generally defined as the absence of discrimination based on certain individual or group characteristics (e.g., race or gender), and several conditions and definitions have been developed in this area to assess whether algorithms are biased with respect to certain attributes (Coston et al, 2020; Mehrabi et al, 2019). Unfortunately, there is ongoing debate about the achievability, relative benefits of these conditions and risks that may result from fairness‐correcting methods, sometimes for the very population they were intended to protect (Coston et al, 2020; Kleinberg et al, 2016; Mehrabi et al, 2019).…”

“…In this context, fairness is generally defined as the absence of discrimination based on certain individual or group characteristics (e.g., race or gender), and several conditions and definitions have been developed in this area to assess whether algorithms are biased with respect to certain attributes (Coston et al, 2020; Mehrabi et al, 2019). Unfortunately, there is ongoing debate about the achievability, relative benefits of these conditions and risks that may result from fairness‐correcting methods, sometimes for the very population they were intended to protect (Coston et al, 2020; Kleinberg et al, 2016; Mehrabi et al, 2019).…”