You Can’t See What You Can’t See: Experimental Evidence for How Much Relevant Information May Be Missed Due to Google’s Web Search Personalisation

Luczak‐Roesch, Markus

doi:10.1007/978-3-030-34971-4_17

Cited by 9 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Robertson et al [137] 2019 Direct scrape Search Google Bing Ali et al [5] 2019 Carrier puppet Advertising Facebook Cano-Orón [24] 2019 Direct scrape Search Google Lai and Luczak-Roesch [97] 2019 Crowdsourcing Search Google Hussein et al [82] 2020 Sock puppets Search YouTube Ribeiro et al [134] 2020 below the links on search result pages. They found that partisan cues in the snippet text tend to amplify the partisanship of the original web page, an effect that was consistent across query topics, left-and right-leaning queries, and different types of web pages (e.g.…”

Section: Distortion In Searchmentioning

confidence: 99%

“…These findings could be limited to socio-political queries -as the earliest personalization audit showed, the degree of personalization varies across query categories [73]. Rather than queries for political candidates and topics, one study in New Zealand by Lai and Luczak-Roesch [97] used queries that public officials "might perform in the course of their everyday work duties. " Asking 30 participants to rate the relevance of personalized and nonpersonalized results, their findings suggest that up to 20% of relevant results were removed due to personalization.…”

Section: Distortion In Searchmentioning

confidence: 99%

“…Future audits should strive for user-centered audits, and may benefit from scoping to specific use cases. For example, such audits can build on the work of Lai and Luczak-Roesch [97] which scoped to public officials' use of search algorithms, and the work of Mustafaraj et al [120] which outlines methods for voter-centric audits. Also, Fischer et al [59] scoped their audit to local news, a topic which deserves more audit attention given the current crisis in local journalism and its threat to impacted communities (e.g.…”

Section: Remaining Work: Distortionmentioning

confidence: 99%

See 2 more Smart Citations

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Bandy

2021

Preprint

View full text Add to dashboard Cite

While algorithm audits are growing rapidly in commonality and public importance, relatively little scholarly work has gone toward synthesizing prior work and strategizing future research in the area. This systematic literature review aims to do just that, following PRISMA guidelines in a review of over 500 English articles that yielded 62 algorithm audit studies. The studies are synthesized and organized primarily by behavior (discrimination, distortion, exploitation, and misjudgement), with codes also provided for domain (e.g. search, vision, advertising, etc.), organization (e.g. Google, Facebook, Amazon, etc.), and audit method (e.g. sock puppet, direct scrape, crowdsourcing, etc.). The review shows how previous audit studies have exposed public-facing algorithms exhibiting problematic behavior, such as search algorithms culpable of distortion and advertising algorithms culpable of discrimination. Based on the studies reviewed, it also suggests some behaviors (e.g. discrimination on the basis of intersectional identities), domains (e.g. advertising algorithms), methods (e.g. code auditing), and organizations (e.g. Twitter, TikTok, LinkedIn) that call for future audit attention. The paper concludes by offering the common ingredients of successful audits, and discussing algorithm auditing in the context of broader research working toward algorithmic justice.

show abstract

Section: Distortion In Searchmentioning

confidence: 99%

Section: Distortion In Searchmentioning

confidence: 99%

Section: Remaining Work: Distortionmentioning

confidence: 99%

See 1 more Smart Citation

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Bandy

2021

Preprint

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

“…One potential concern relates to the reproducibility of searches using search engines such as Google 9‐11 . That is, unlike database searches, web searches may return different results depending on the user, such that the search process is not fully reproducible 1,9‐13 . While this issue is widely recognised in theory, and some studies indicate it is a real concern, 14 detailed data on the extent of the variation are lacking 9,12,13 …”

Section: Introductionmentioning

confidence: 99%

What you see depends on where you sit: The effect of geographical location on web‐searching for systematic reviews: A case study

Cooper

Lorenc

Schauberger³

2021

Research Synthesis Methods

View full text Add to dashboard Cite

There is limited guidance on how to web‐search in systematic reviews and concern relates to the reproducibility of searches using search engines such as Google. The aim of this paper is to address one potential source of variation in Google searches: does the geographical location of a researcher affect Google search returns? Using a virtual private network, we ran the same web‐search for the medical technology Dasatinib in 12 different countries. Two researchers independently extracted the search returns by country organised by page rank. We compared: C1. any difference in the items returned by Google searches between countries and C2. any difference in the page rank of items returned between countries. Searches were undertaken on Monday September 28th 2020. From 12 countries, 43 items were identified. For C1: 19 items were common to all 12 countries. Twenty‐four items were missed by searches in some countries. This means that there were differences in search returns between countries. For C2: a randomised trial reported by Raddich et al was the first search return for all countries. All other items, common to all countries, varied in their page‐rank. We find that geographic location would appear to influence Google search returns based on the findings of this case study. The findings suggest that recording the location of the researcher undertaking web‐searching may now be an important factor to report alongside detail on steps taken to minimise personalisation of web‐searches covered by recent guidance. This finding also has implications for stopping‐rules.

show abstract

Is googling risky? A study on risk perception and experiences of adverse consequences in web search

Häußler

Schultheiß

Lewandowski

2023

Asso for Info Science & Tech

View full text Add to dashboard Cite

Search engines, such as Google, have a considerable impact on society. Therefore, undesirable consequences, such as retrieving incorrect search results, pose a risk to users. Although previous research has reported the adverse outcomes of web search, little is known about how search engine users evaluate those outcomes. In this study, we show which aspects of web search are perceived as risky using a sample (N = 3884) representative of the German Internet population. We found that many participants are often concerned with adverse consequences immediately appearing on the search engine result page. For example, 45.2% of respondents are concerned about retrieving incorrect information. In contrast, consequences with a delayed impact are rarely perceived as a risk. Moreover, participants' experiences with adverse consequences are directly related to their risk perception. Our results demonstrate that people perceive risks related to web search. In addition to our study, there is a need for more independent research on the possible detrimental outcomes of web search to monitor and mitigate risks. Apart from risks for individuals, search engines with a massive number of users have an extraordinary impact on society; therefore, the acceptable risks of web search should be discussed.

show abstract

You Can’t See What You Can’t See: Experimental Evidence for How Much Relevant Information May Be Missed Due to Google’s Web Search Personalisation

Cited by 9 publications

References 23 publications

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits

What you see depends on where you sit: The effect of geographical location on web‐searching for systematic reviews: A case study

Is googling risky? A study on risk perception and experiences of adverse consequences in web search

Contact Info

Product

Resources

About