Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making

Abstract

A growing literature on human-AI decision-making investigates strategies for combining human judgment with statistical models to improve decision-making. Research in this area often evaluates proposed improvements to models, interfaces, or workflows by demonstrating improved predictive performance on "ground truth" labels. However, this practice overlooks a key difference between human judgments and model predictions. Whereas humans reason about broader phenomena of interest in a decision -including latent constructs that are not directly observable, such as disease status, the "toxicity" of online comments, or future "job performance"predictive models target proxy labels that are readily available in existing datasets. Predictive models' reliance on simplistic proxies makes them vulnerable to various sources of statistical bias. In this paper, we identify five sources of target variable bias that can impact the validity of proxy labels in human-AI decision-making tasks. We develop a causal framework to disentangle the relationship between each bias and clarify which are of concern in specific human-AI decision-making tasks. We demonstrate how our framework can be used to articulate implicit assumptions made in prior modeling work, and we recommend evaluation strategies for verifying whether these assumptions hold in practice. We then leverage our framework to re-examine the designs of prior human subjects experiments that investigate human-AI decision-making, finding that only a small fraction of studies examine factors related to target variable bias. We conclude by discussing opportunities to better address target variable bias in future research.

A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity

“…Instead they are inferred indirectly via proxies: measurements of properties that are observed in the data available to a model. The process of defining proxy variables for a construct of interest necessarily involves making simplifying assumptions, and there is often a considerable conceptual distance between ML proxies and the ways human decision-makers think about the targeted construct (Green and Chen 2021;Guerdan et al 2023;Jacobs and Wallach 2021;Kawakami et al 2022). In other words, O H (X, a) ̸ = O M (X, a).…”

“…Much prior work has studied settings where the ML model outperforms the human decision-maker. These studies are frequently focused on tasks where there are no reasons to expect upfront that the human and the ML model will have complementary strengths (Bansal et al 2021;Guerdan et al 2023;Holstein and Aleven 2021;Lurie and Mulligan 2020). For example, some experimental studies employ untrained crowdworkers on tasks that require extensive domain expertise, without which there is no reason to expect that novices would have complementary strengths (Fogliato, Chouldechova, and Lipton 2021;Lurie and Mulligan 2020;Rastogi et al 2022).…”

“…For example, some experimental studies employ untrained crowdworkers on tasks that require extensive domain expertise, without which there is no reason to expect that novices would have complementary strengths (Fogliato, Chouldechova, and Lipton 2021;Lurie and Mulligan 2020;Rastogi et al 2022). Other experimental studies are designed in ways that artificially constrain human performance-for instance, by eliminating the possibility that humans and ML systems have access to complementary information (Guerdan et al 2023). Meanwhile studies on human-ML decisionmaking in real-world settings such as healthcare (Tschandl et al 2020;Patel et al 2019) sometimes demonstrate better human-ML team performance than either agent alone.…”