Studying Up Machine Learning Data

Abstract

Research in machine learning (ML) has argued that models trained on incomplete or biased datasets can lead to discriminatory outputs. In this commentary, we propose moving the research focus beyond bias-oriented framings by adopting a power-aware perspective to "study up" ML datasets. This means accounting for historical inequities, labor conditions, and epistemological standpoints inscribed in data. We draw on HCI and CSCW work to support our argument, critically analyze previous research, and point at two co-existing lines of work within our research community \,---\,one bias-centered, the other power-aware. We highlight the need for dialogue and cooperation in three areas: data quality, data work, and data documentation. In the first area, we argue that reducing societal problems to "bias" misses the context-based nature of data. In the second one, we highlight the corporate forces and market imperatives involved in the labor of data workers that subsequently shape ML datasets. Finally, we propose expanding current transparency-oriented efforts in dataset documentation to reflect the social contexts of data design and production.

Race-neutral vs race-conscious: Using algorithmic methods to evaluate the reparative potential of housing programs

“…The emerging field of algorithmic reparations aims to address such issues of algorithmic fairness and discrimination by developing new methods that consider the structural conditions of oppression and inequality. This is in line with work that advocates for moving away from the narrow idea of “bias” toward a more robust conceptual, computational, and historical modeling of “power” in algorithms and machine learning (D’Ignazio and Klein, 2020; Miceli et al, 2022). Davis et al (2021) suggest that a reparative approach using algorithms can contribute to redressing past harms by utilizing the principles of intersectionality and reparations.…”

AI Companion Robot Data Sharing: Preferences of an Online Cohort and Policy Implications

“…Older adults' interests are thus far not well represented in the larger AI and data privacy policy discourse (Stypińska, 2021; WHO, 2022). It is critical that these interests be surfaced and represented given the diverse values, demand for data, its commercialization, and the range of harms that have been identified among other marginalized communities (Green, 2021; Greene et al, 2019; Hoffmann, 2019; Miceli et al, 2022). Optional comments offered by 38% of our participants provide some additional insight into concerns about artificial companion robots that are largely consistent with those expressed by gerontechnologists and geriatric care professionals (Berridge et al, 2021; Wangmo et al, 2019).…”

The benefits and dangers of using machine learning to support making legal predictions

“…When machine learning systems use such data sets, biases which are very difficult to detect, may occur. The end result may be that certain groups suffer discrimination (Miceli et al, 2022). It should be the goal of AI and Law researchers to minimize the manner in which machine learning uses “compromised data” to make automated decisions that are discriminatory.…”