On the Social and Technical Challenges of Web Search Autosuggestion Moderation

Abstract

Past research shows that users benefit from systems that support them in their writing and exploration tasks. The autosuggestion feature of Web search engines is an example of such a system: It helps users in formulating their queries by offering a list of suggestions as they type. Autosuggestions are typically generated by machine learning (ML) systems trained on a corpus of search logs and document representations. Such automated methods can become prone to issues that result in problematic suggestions that are biased, racist, sexist or in other ways inappropriate. While current search engines have become increasingly proficient at suppressing such problematic suggestions, there are still persistent issues that remain. In this paper, we reflect on past efforts and on why certain issues still linger by covering explored solutions along a prototypical pipeline for identifying, detecting, and addressing problematic autosuggestions. To showcase their complexity, we discuss several dimensions of problematic suggestions, difficult issues along the pipeline, and why our discussion applies to the increasing number of applications beyond web search that implement similar textual suggestion features. By outlining persistent social and technical challenges in moderating web search suggestions, we provide a renewed call for action.

Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models

“…ILPS and SeT both require templates in calculating scores. We thus carefully construct a list of templates (Table 4) that covers multiple grammatical and semantic variations, inspired by work investigating harmful search automatic suggestions (Hazen et al, 2020). We find that different model structure requires different templates in order to bring up stereotypes that correlate with human data.…”

Algorithmic probing: Prompting offensive Google results and their moderation

“…Platform 'suppression' strategies are of interest in this regard. There are such practices as maintaining block lists which are lists of highly offensive terms that will return no result as well as the so-called n-strike rule which extends the working of the lists to phrases (Hazen et al, 2020). Indeed, rather than 'fixes', the work is in the realm of patching and maintenance.…”

Analyzing Stereotypes in Generative Text Inference Tasks

“…With the collected human judgments, we first investigate which models and categories lead to stereotyped inferences, and the degree to which the invoked stereotypes are negative. It is well established that stereotypes are both an individual phenomenon-something that resides in the heads of individual people-as well as a cultural phenomenon-that "[sterotypes] exist also in 'the fabric of society' itself" (Stangor and Schaller, 2012), and as such who the annotators are matters (Hovy and Spruit, 2016;Jørgensen et al, 2015;Hazen et al, 2020). In view of this, part of our analysis specifically considers how individual annotators' perceptions of stereotypes may vary.…”

“…: annotators choose whether the hypothesis conforms to stereotypes they know; options are yes, no, or maybe. People's perceptions on whether a hypothesis is stereotypical or problematical are highly subjective (Hazen et al, 2020), and one research question we seek to answer is how annotators' levels of agreement may vary for different target categories (see §4.2). Overall, stereotypes can harmful even when positive; the nurturing stereotype of women is used to justify exclusion from professional settings (Tinsley et al, 2009), and, for women who do not conform to the stereotype, can lead to increased sexual harassment (Leskinen et al, 2015).…”