Participation Is not a Design Fix for Machine Learning

Sloane, Mona; Moss, Emanuel; Awomolo, Olaitan; Forlano, Laura

doi:10.1145/3551624.3555285

Cited by 66 publications

(32 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, status quo forums for participation may be structurally inaccessible and participation itself may carry disproportionate risks for certain communities, especially undocumented communities. Thus, equitable participation requires developing modes of participation that enable transparency, generative friction, and meaningful forms of knowledge exchange (Katell et al, 2020; Sloane et al, 2020), prioritizing the needs of the margins.…”

Section: Participatory Ai and Algorithmic Accountabilitymentioning

confidence: 99%

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Rakova,

Shelby,

2023

Big Data & Society

View full text Add to dashboard Cite

Power and information asymmetries between people and digital technology companies are further legitimized through contractual agreements that fail to provide meaningful consent and contestability. In particular, the Terms-of-Service (ToS) agreement, is a contract of adhesion where companies effectively set the terms and conditions of the contract. Whereas, ToS reinforce existing structural inequalities, we seek to enable an intersectional accountability mechanism grounded in the practice of algorithmic reparation. Building on existing critiques of ToS in the context of algorithmic systems, we return to the roots of contract theory by recentering notions of agency and mutual assent. We evolve a multipronged intervention we frame as the Terms-we-Serve-with (TwSw) social, computational, and legal framework. The TwSw is a new social imaginary centered on: (1) co-constitution of user agreements, through participatory mechanisms; (2) addressing friction, leveraging the fields of design justice and critical design in the production and resolution of conflict; (3) enabling refusal mechanisms, reflecting the need for a sufficient level of human oversight and agency including opting out; (4) complaint and algorithmic harms reporting, through a feminist studies lens and open-sourced computational tools; and (5) disclosure-centered mediation, to disclose, acknowledge, and take responsibility for harm, drawing on the field of medical law. We further inform our analysis through an exploratory design workshop with a South African gender-based violence reporting AI startup. We derive practical strategies for communities, technologists, and policy-makers to leverage a relational approach to algorithmic reparation and propose there's a need for a radical restructuring of the “take-it-or-leave-it” ToS agreement.

show abstract

Section: Participatory Ai and Algorithmic Accountabilitymentioning

confidence: 99%

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Rakova,

Shelby,

2023

Big Data & Society

View full text Add to dashboard Cite

show abstract

“…pretrained models may not solve problems for different local contexts. [32] Instead, local researchers must be provided with the tools to develop local solutions to local issues. [33] This echoes the still unsolved problem of drug development for neglected tropical diseases, where local bottom-up approaches hold great promise.…”

Section: Inclusive Machine Learning Researchmentioning

confidence: 99%

Growing pains: Reacting to negative impacts of deep learning on machine learning for chemistry

Probst

2022

Preprint

View full text Add to dashboard Cite

While the introduction of practical deep learning has driven progress across scientific fields, recent research highlighted that deep learning has potential negative impacts on the scientific community and society as a whole. An ever-growing need for more computational resources may exacerbate the concentration of funding and the exclusiveness of research between countries, sectors, and institutions. Here, I introduce recent concerns and considerations of the machine learning research community and present potential solutions.

show abstract

“…Participatory approaches have been enacted across each stage of ML design and development-from problem formulation to model evaluation-and include collaborative approaches to construct datasets [62,73], design and validate ML algorithms [57,72], and guide advocacy for algorithmic accountability [47,63]. At the same time, several authors have raised concerns about "participation-washing" [70], cooptation of participatory work [7], and the limited evidence across Participatory ML projects of equitable partnerships with participants [24,26,33].…”

Section: Introductionmentioning

confidence: 99%

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Cooper,

Zafiroglu

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Participatory machine learning (ML) encourages the inclusion of end users and people affected by ML systems in design and development processes. We interviewed 18 participation brokers-individuals who facilitate such inclusion and transform the products of participants' labour into inputs for an ML artefact or system-across a range of organisational settings and project locations. Our findings demonstrate the inherent challenges of integrating messy contextual information generated through participation with the structured data formats required by ML workflows and the uneven power dynamics in project contexts. We advocate for evolution in the role of brokers to more equitably balance value generated in Participatory ML projects for design and development teams with value created for participants. To move beyond 'fitting' participation to existing processes and empower participants to envision alternative futures through ML, brokers must become educators and advocates for end users, while attending to frustration and dissent from indirect stakeholders.

show abstract

Participation Is not a Design Fix for Machine Learning

Cited by 66 publications

References 16 publications

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Terms-we-serve-with: Five dimensions for anticipating and repairing algorithmic harm

Growing pains: Reacting to negative impacts of deep learning on machine learning for chemistry

From Fitting Participation to Forging Relationships: The Art of Participatory ML

Contact Info

Product

Resources

About