Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors

Shen, Hong; DeVos, Alicia; Eslami, Motahhare; Holstein, Kenneth

doi:10.1145/3479577

Cited by 76 publications

(61 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to posthoc audits and evaluations, certificates of robustness provide algorithmic guarantees on the performance of models under certain conditions [76]. Human-inthe-loop auditing processes leverage everyday users to provide further protection once a system is deployed [63], and we heard from participants in both of our studies that they believed IMCs would be valuable for record-keeping and organizational alignment throughout deployment.…”

Section: Background and Related Workmentioning

confidence: 99%

Interactive Model Cards: A Human-Centered Approach to Model Documentation

Crisan,

Drouhard,

Vig

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML). However, the documentation intended to convey the model's details and appropriate use is tailored primarily to individuals with ML or NLP expertise. To address this gap, we conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves. Our investigation consists of an initial conceptual study with experts in ML, NLP, and AI Ethics, followed by a separate evaluative study with non-expert analysts who use ML models in their work. Using a semi-structured interview format coupled with a think-aloud protocol, we collected feedback from a total of 30 participants who engaged with different versions of standard and interactive model cards. Through a thematic analysis of the collected data, we identified several conceptual dimensions that summarize the strengths and limitations of standard and interactive model cards, including: stakeholders; design; guidance; understandability & interpretability; sensemaking & skepticism; and trust & safety. Our findings demonstrate the importance of carefully considered design and interactivity for orienting and supporting non-expert analysts using deep learning models, along with a need for consideration of broader sociotechnical contexts and organizational dynamics. We have also identified design elements, such as language, visual cues, and warnings, among others, that support interactivity and make non-interactive content accessible. We summarize our findings as design guidelines and discuss their implications for a human-centered approach towards AI/ML documentation.CCS Concepts: • Computing methodologies → Natural language processing; • Human-centered computing → Visualization;Human computer interaction (HCI); Interaction design process and methods.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Interactive Model Cards: A Human-Centered Approach to Model Documentation

Crisan,

Drouhard,

Vig

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Future work is needed to explore ways to build sustainable, diverse, interdisciplinary communities of practice. Toolkit developers interested in pursuing this vision may be able to draw lessons from other open-source community-building efforts [21,47], or from recent work exploring ways to support collective algorithm auditing [6,22,24,97]. In short, we advocate for future fairness toolkits to position themselves as socio-technical systems that enable more collaborative approaches to ML fairness practice.…”

Section: Fostering Interdisciplinary Communication and Collaborationmentioning

confidence: 99%

Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits

Deng,

Nagireddy,

Lee

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent years have seen the development of many open-source ML fairness toolkits aimed at helping ML practitioners assess and address unfairness in their systems. However, there has been little research investigating how ML practitioners actually use these toolkits in practice. In this paper, we conducted the first in-depth empirical exploration of how industry practitioners (try to) work with existing fairness toolkits. In particular, we conducted think-aloud interviews to understand how participants learn about and use fairness toolkits, and explored the generality of our findings through an anonymous online survey. We identified several opportunities for fairness toolkits to better address practitioner needs and scaffold them in using toolkits effectively and responsibly. Based on these findings, we highlight implications for the design of future open-source fairness toolkits that can support practitioners in better contextualizing, communicating, and collaborating around ML fairness efforts.

show abstract

“…First, it is widely accepted that algorithms' opacity (Diakopoulos & Koliska, 2017)-or what Pasquale (2015) calls the "black box" of algorithmic decision makingmakes it difficult to curtail platform power, which has motivated a growing body of empirical research interested in studying algorithms from the outside. This includes methods such as "reverse engineering" (Diakopoulos, 2015), "scraping audits" (Sandvig et al, 2014), "everyday algorithm auditing" (Shen et al, 2021), small-scale observation (Bucher, 2012), and systematic large-scale observation (Rieder et al, 2018). Second, debates around how to hold the media accountable in general, and social media in particular, tend to focus on calls for greater transparency for regulatory inspection (Diakopoulos, 2016;Pasquale, 2015).…”

Section: Social Media Recommender Systems Exposure Diversity and Platform Observabilitymentioning

confidence: 99%

What’s “Up Next”? Investigating Algorithmic Recommendations on YouTube Across Issues and Over Time

Matamoros-Fernández

Gray

Bartolo

et al. 2021

MaC

View full text Add to dashboard Cite

YouTube’s “up next” feature algorithmically selects, suggests, and displays videos to watch after the one that is currently playing. This feature has been criticized for limiting users’ exposure to a range of diverse media content and information sources; meanwhile, YouTube has reported that they have implemented various technical and policy changes to address these concerns. However, there is little publicly available data to support either the existing concerns or YouTube’s claims of having addressed them. Drawing on the idea of “platform observability,” this article combines computational and qualitative methods to investigate the types of content that the algorithms underpinning YouTube’s “up next” feature amplify over time, using three keyword search terms associated with sociocultural issues where concerns have been raised about YouTube’s role: “coronavirus,” “feminism,” and “beauty.” Over six weeks, we collected the videos (and their metadata, including channel IDs) that were highly ranked in the search results for each keyword, as well as the highly ranked recommendations associated with the videos. We repeated this exercise for three steps in the recommendation chain and then examined patterns in the recommended videos (and the channels that uploaded the videos) for each query and their variation over time. We found evidence of YouTube’s stated efforts to boost “authoritative” media outlets, but at the same time, misleading and controversial content continues to be recommended. We also found that while algorithmic recommendations offer diversity in videos over time, there are clear “winners” at the channel level that are given a visibility boost in YouTube’s “up next” feature. However, these impacts are attenuated differently depending on the nature of the issue.

show abstract

Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors

Cited by 76 publications

References 60 publications

Interactive Model Cards: A Human-Centered Approach to Model Documentation

Interactive Model Cards: A Human-Centered Approach to Model Documentation

Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits

What’s “Up Next”? Investigating Algorithmic Recommendations on YouTube Across Issues and Over Time

Contact Info

Product

Resources

About