Toward User-Driven Algorithm Auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior

DeVos, Alicia; Dhabalia, Aditi; Shen, Hong; Holstein, Kenneth; Eslami, Motahhare

doi:10.1145/3491102.3517441

Cited by 43 publications

(29 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, patterns of demeaning imagery have been found in hateful natural language predictions Alienating social groups "[lack of representation] further promotes the idea that you don't belong and perpetuates the sense of alienation. " [61] Denying people opportunity to self-identify "It's definitely frustrating having [classifiers] get integral parts of my identity wrong. And I find it frustrating that these sorts of apps only tend to recognize two binary genders. "…”

Section: Stereotypingmentioning

confidence: 99%

“…This dimension of representational harm diminishes human dignity [120] and is especially likely when "a system fails to recognize the injustices suffered by specific social groups" [104]. A study of user-elicited identification of harms in image search describes the impacts of such failures as "further promot[ing] the idea that you don't belong and perpetuat[ing] a sense of alienation" [61] (p. 8).…”

Section: Stereotypingmentioning

confidence: 99%

“…Diminished health and well-being can arise through purposeful behavioral exploitation [18,184] or emotional manipulation [177], whereby algorithmic designs exploit user behavior, through safety failures (e.g., collisions) [57], or when algorithmic systems make incorrect health inferences [139]. They can lead to both physical harms [61,69,120,160], emotional harms, such as distress [208], dignity loss [38,120], and misgendering [106], and reputational harms [132,160]. Diminished health and well-being may accompany other identified sociotechnical harms -for example, experiences of representational harms, including algorithmic annihilation [8] or the internalization of stereotypes, may spark other emotional or psychological effects, such as "epistemic doubt" [205], which affects overall health.…”

Section: Interpersonal Harms: Technological Affordances Adversely Sha...mentioning

confidence: 99%

“…[151] (p. 30) [4]. As algorithmic technologies can "foreclose alternative ways of understanding the world and restricting imaginations about possible futures" (p. 162) [164], the nature of their harm can encompass adverse cultural such as systemic erasure [61], eurocentric ideas being exported to Global South [67,129], harmful cultural beliefs [66], such as normalizing a culture of non-consensual sexual activity [123], or proliferating false ideas about cultural groups [68,167].…”

Section: Cultural Harmsmentioning

confidence: 99%

“…The taxonomy offered here is a starting point for surfacing harms. We expect and hope it will evolve as research progresses, particularly in terms of user and community-driven research methodologies (e.g., [58,61,79]). The CHI community is well positioned to strengthen understandings of harm.…”

Section: Advancing Sociotechnical Research On Harmsmentioning

confidence: 99%

See 4 more Smart Citations

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Shelby¹,

Rismani²,

Henne³

et al. 2022

Preprint

View full text Add to dashboard Cite

Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems arising at the micro-, meso-, and macro-levels of society. We present an applied taxonomy of sociotechnical harms to support more systematic surfacing of potential harms in algorithmic systems. Based on a scoping review of computing research (n=172), we identified five major themes related to sociotechnical harms -representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms -and sub-themes. We describe these categories and conclude with a discussion of challenges and opportunities for future research.

show abstract

Section: Stereotypingmentioning

confidence: 99%

Section: Stereotypingmentioning

confidence: 99%

Section: Interpersonal Harms: Technological Affordances Adversely Sha...mentioning

confidence: 99%

Section: Cultural Harmsmentioning

confidence: 99%

Section: Advancing Sociotechnical Research On Harmsmentioning

confidence: 99%

See 3 more Smart Citations

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Shelby¹,

Rismani²,

Henne³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Data Management

Bialkova

2024

The Rise of AI User Applications

View full text Add to dashboard Cite

Auditing of AI: Legal, Ethical and Technical Approaches

Mökander

2023

DISO

View full text Add to dashboard Cite

AI auditing is a rapidly growing field of research and practice. This review article, which doubles as an editorial to Digital Society’s topical collection on ‘Auditing of AI’, provides an overview of previous work in the field. Three key points emerge from the review. First, contemporary attempts to audit AI systems have much to learn from how audits have historically been structured and conducted in areas like financial accounting, safety engineering and the social sciences. Second, both policymakers and technology providers have an interest in promoting auditing as an AI governance mechanism. Academic researchers can thus fill an important role by studying the feasibility and effectiveness of different AI auditing procedures. Third, AI auditing is an inherently multidisciplinary undertaking, to which substantial contributions have been made by computer scientists and engineers as well as social scientists, philosophers, legal scholars and industry practitioners. Reflecting this diversity of perspectives, different approaches to AI auditing have different affordances and constraints. Specifically, a distinction can be made between technology-oriented audits, which focus on the properties and capabilities of AI systems, and process-oriented audits, which focus on technology providers’ governance structures and quality management systems. The next step in the evolution of auditing as an AI governance mechanism, this article concludes, should be the interlinking of these available—and complementary—approaches into structured and holistic procedures to audit not only how AI systems are designed and used but also how they impact users, societies and the natural environment in applied settings over time.

show abstract

Toward User-Driven Algorithm Auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior

Cited by 43 publications

References 29 publications

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Data Management

Auditing of AI: Legal, Ethical and Technical Approaches

Contact Info

Product

Resources

About