When More Data Lead Us Astray: Active Data Acquisition in the Presence of Label Bias

Abstract

An increased awareness concerning risks of algorithmic bias has driven a surge of efforts around bias mitigation strategies. A vast majority of the proposed approaches fall under one of two categories: (1) imposing algorithmic fairness constraints on predictive models, and (2) collecting additional training samples. Most recently and at the intersection of these two categories, methods that propose active learning under fairness constraints have been developed. However, proposed bias mitigation strategies typically overlook the bias presented in the observed labels. In this work, we study fairness considerations of active data collection strategies in the presence of label bias. We first present an overview of different types of label bias in the context of supervised learning systems. We then empirically show that, when overlooking label bias, collecting more data can aggravate bias, and imposing fairness constraints that rely on the observed labels in the data collection process may not address the problem. Our results illustrate the unintended consequences of deploying a model that attempts to mitigate a single type of bias while neglecting others, emphasizing the importance of explicitly differentiating between the types of bias that fairness-aware algorithms aim to address, and highlighting the risks of neglecting label bias during data collection.

An Operational Perspective to Fairness Interventions: Where and How to Intervene

“…This framework overall aims to capture the major considerations in operationalizing fairness that are quantifiable and enable benchmarking to some extent, as we believe that this helps practitioners decide how to make trade-offs between the pillars. We remark that while these are the classical categorizations in ML pipelines, there are still applications that use group data in ways that fall outside of these categories (for instance at the data collection step [10,39], we also propose one such method in the next section at the feature selection step). These methods should be considered, but our work focuses on bringing more structure and order to the majority of fairness intervention work in the highlighted categories [28].…”

An Anti-Subordination Approach to Fair Classification

“…Active learning-based data collection approaches assume that labels can be queried for each sample at a fixed cost [7,25,1,34]. Unlike these approaches, our framework uses constraints on false positives and assumes non-constant labeling costs that depend on the outcome and the classifier.…”

“…Other methods of data collection, e.g. using human annotators [25] or augmenting using third-party data [15] are often infeasible as they can only provide proxies for the true outcomes, which themselves can encode social biases [35].…”