Situated algorithms: a sociotechnical systemic approach to bias

Abstract

Purpose The purpose of this paper is to propose that in order to tackle the question of bias in algorithms, a systemic, sociotechnical and holistic perspective is needed. With reference to the term “algorithmic culture,” the interconnectedness and mutual shaping of society and technology are postulated. A sociotechnical approach requires translational work between and across disciplines. This conceptual paper undertakes such translational work. It exemplifies how gender and diversity studies, by bringing in expertise on addressing bias and structural inequalities, provide a crucial source for analyzing and mitigating bias in algorithmic systems. Design/methodology/approach After introducing the sociotechnical context, an overview is provided regarding the contemporary discourse around bias in algorithms, debates around algorithmic culture, knowledge production and bias identification as well as common solutions. The key concepts of gender studies (situated knowledges and strong objectivity) and concrete examples of gender bias then serve as a backdrop for revisiting contemporary debates. Findings The key concepts reframe the discourse on bias and concepts such as algorithmic fairness and transparency by contextualizing and situating them. The paper includes specific suggestions for researchers and practitioners on how to account for social inequalities in the design of algorithmic systems. Originality/value A systemic, gender-informed approach for addressing the issue is provided, and a concrete, applicable methodology toward a situated understanding of algorithmic bias is laid out, providing an important contribution for an urgent multidisciplinary dialogue.

Investigating Bias in Facial Analysis Systems: A Systematic Review

“…According to such evidence, it is clear that algorithmic audits play a fundamental role in informing strategies for engaging both researchers and corporations in effectively addressing algorithmic bias. Mitigating algorithmic bias is a difficult task that requires a systemic, sociotechnical and holistic perspective [56].The publication of the Gender Shades study [18], for instance, not only played a significant role in sparking interest in gender and ethnicity classification but also motivated the research community to investigate bias and discrimination in facial analysis algorithms and systems in other domains, for instance, detecting bias against older adults in clinical settings [6] and bias against children in state-of-the-art pedestrian detection algorithms. In addition, it had a major positive effect on commercial gender classification services.…”

Dashboard stories: How narratives told by predictive analytics reconfigure roles, risk and sociality in education

“…Importantly, data justice and feminist scholarship have shown that some users will invariably appropriate the system in ‘other’ ways (e.g. D’Ignazio and Klein, 2020; Draude et al., 2019). 2 To understand how predictive analytics is reconfiguring education, it is important to also highlight ambivalences, and trace how predictive tools are embedded into unexpected or unforeseen practices.…”

Social Machines

“…In der Nutzung von Social Machines treten solche Setzungen dann zumeist als technologische Eigengesetzlichkeiten auf (Code is Law) bzw. sind für die Nutzenden nicht verständlich oder sichtbar [9]. Diesem muss durch entsprechende Gestaltungsverfahren entgegengewirkt werden.…”