The Effect of Content-Equivalent Near-Duplicates on the Evaluation of Search Engines

Fröbe, Maik; Bittner, Jan Philipp; Potthast, Martin; Hagen, Matthias

doi:10.1007/978-3-030-45442-5_2

Cited by 12 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This allows us to develop and maintain services such as the ChatNoir web search engine [Be18], the argument search engine args.me [Wa17], but also facilitates individual research projects which rely on utilizing the bundled computational power and storage capacity of the capable Hadoop, Kubernetes, and Ceph clusters. A few recent examples are the analysis of massive query logs [Bo20], a detailed inquiry into near-duplicates in web search [Fr20b,Fr20a], the preparation, parsing, and assembly of large NLP corpora [Be20,Ke20], and the exploration of large transformer models for argument retrieval [AP20].…”

Section: Discussionmentioning

confidence: 99%

Web Archive Analytics

Völske,

Bevendorff,

Kiesel

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Web archive analytics is the exploitation of publicly accessible web pages and their evolution for research purposes-to the extent organizationally possible for researchers. In order to better understand the complexity of this task, the first part of this paper puts the entirety of the world's captured, created, and replicated data (the "Global Datasphere") in relation to other important data sets such as the public internet and its web pages, or what is preserved thereof by the Internet Archive. Recently, the Webis research group, a network of university chairs to which the authors belong, concluded an agreement with the Internet Archive to download a substantial part of its web archive for research purposes. The second part of the paper in hand describes our infrastructure for processing this data treasure: We will eventually host around 8 PB of web archive data from the Internet Archive and Common Crawl, with the goal of supplementing existing large scale web corpora and forming a non-biased subset of the 30 PB web archive at the Internet Archive.

show abstract

Section: Discussionmentioning

confidence: 99%

Web Archive Analytics

Völske,

Bevendorff,

Kiesel

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Strategic providers can create duplicates or near-duplicates-possibly hard to automatically identifyof their existing offerings in a ranking system. Since certain fair ranking mechanisms may try to ensure benefits for all listed items, providers with more copies of same items stand to gain more benefits [57,43]. We give such an example in Appendix Table 4.…”

Section: Strategic Behaviormentioning

confidence: 99%

“…on the RS to improve their ranking [175], influencers on social network sites could try to hijack popular trends [65,32]. Providers can even strategically exploit the deployed fair ranking mechanisms to extract more benefits [57,43]. Not factoring in such strategic behavior could impact ranking and recommendation systems, and especially the performance of fair ranking mechanisms.…”

Section: Strategic Behaviormentioning

confidence: 99%

See 1 more Smart Citation

Fair ranking: a critical review, challenges, and future directions

Patro¹,

Porcaro²,

Mitchell³

et al. 2022

Preprint

View full text Add to dashboard Cite

Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking" research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such an approach misses: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.

show abstract

“…Therefore, they introduce the so-called novelty principle, which states that a document, though relevant in isolation, is irrelevant if it is a near-duplicate to a document the user has already seen on the search engine result page. Especially on web crawls with many near-duplicates, the novelty principle has a non-negligible impact on evaluating search engines [14]. E.g., applying the novelty principle on the runs submitted to the Terabyte track 2004 decreases mean average precision scores by 20% on average [6].…”

Section: Risk: Evaluation Of Search Enginesmentioning

confidence: 99%

The Impact of Main Content Extraction on Near-Duplicate Detection

Fröbe¹,

Hagen²,

Bevendorff³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Commercial web search engines employ near-duplicate detection to ensure that users see each relevant result only once, albeit the underlying web crawls typically include (near-)duplicates of many web pages. We revisit the risks and potential of near-duplicates with an information retrieval focus, motivating that current efforts toward an open and independent European web search infrastructure should maintain metadata on duplicate and near-duplicate documents in its index.Near-duplicate detection implemented in an open web search infrastructure should provide a suitable similarity threshold, a difficult choice since identical pages may substantially differ in parts of a page that are irrelevant to searchers (templates, advertisements, etc.). We study this problem by comparing the similarity of pages for five (main) content extraction methods in two studies on the ClueWeb crawls. We find that the full content of pages serves precision-oriented near-duplicate-detection, while main content extraction is more recall-oriented.

show abstract

The Effect of Content-Equivalent Near-Duplicates on the Evaluation of Search Engines

Cited by 12 publications

References 15 publications

Web Archive Analytics

Web Archive Analytics

Fair ranking: a critical review, challenges, and future directions

The Impact of Main Content Extraction on Near-Duplicate Detection

Contact Info

Product

Resources

About