Quantifying Engagement with Citations on Wikipedia

Abstract

Wikipedia is one of the most visited sites on the Web and a common source of information for many users. As an encyclopedia, Wikipedia was not conceived as a source of original information, but as a gateway to secondary sources: according to Wikipedia's guidelines, facts must be backed up by reliable sources that reflect the full spectrum of views on the topic. Although citations lie at the heart of Wikipedia, little is known about how users interact with them. To close this gap, we built client-side instrumentation for logging all interactions with links leading from English Wikipedia articles to cited references during one month, and conducted the first analysis of readers' interactions with citations. We find that overall engagement with citations is low: about one in 300 page views results in a reference click (0.29% overall; 0.56% on desktop; 0.13% on mobile). Matched observational studies of the factors associated with reference clicking reveal that clicks occur more frequently on shorter pages and on pages of lower quality, suggesting that references are consulted more commonly when Wikipedia itself does not contain the information sought by the user. Moreover, we observe that recent content, open access sources, and references about life events (births, deaths, marriages, etc.) are particularly popular. Taken together, our findings deepen our understanding of Wikipedia's role in a global information economy where reliability is ever less certain, and source attribution ever more vital.

COVID-19 research in Wikipedia

“…Wikipedia articles are created, improved, and maintained by the efforts of the community of volunteer editors (Chen & Roth, 2012;Priedhorsky, Chen, et al, 2007), and they are used in a variety of ways by a wide user base (Lemmerich, Sáez-Trumper, et al, 2019;Piccardi, Redi, et al, 2020;Singer, Lemmerich, et al, 2017). The information Wikipedia contains is generally considered to be of high quality and up to date (Adams et al, 2020;Geiger & Halfaker, 2013;Keegan, Gergle, & Contractor, 2011;Kumar, West, & Leskovec, 2016;Piscopo & Simperl, 2019;Priedhorsky et al, 2007;Smith, 2020), notwithstanding room for improvement and the need for constant knowledge maintenance (Chen & Roth, 2012;Forte, Andalibi, et al, 2018;Lewoniewski, Wę cel, & Abramovich, 2017).…”

“…The information Wikipedia contains is generally considered to be of high quality and up to date (Adams et al, 2020;Geiger & Halfaker, 2013;Keegan, Gergle, & Contractor, 2011;Kumar, West, & Leskovec, 2016;Piscopo & Simperl, 2019;Priedhorsky et al, 2007;Smith, 2020), notwithstanding room for improvement and the need for constant knowledge maintenance (Chen & Roth, 2012;Forte, Andalibi, et al, 2018;Lewoniewski, Wę cel, & Abramovich, 2017). Following Wikipedia's editorial guidelines, the community of editors creates content often relying on scientific and scholarly literature (Arroyo-Machado, Torres-Salinas, et al, 2020;Halfaker, Mansurov, et al, 2018;Nielsen, Mietchen, & Willighagen, 2017), and therefore Wikipedia can be considered a mainstream gateway to scientific information (Heilman, Kemmann, et al, 2011;Laurent & Vickers, 2009;Lewoniewski et al, 2017;Maggio, Willinsky, et al, 2019;Piccardi et al, 2020;Shafee, Masukume, et al, 2017). Unfortunately, few studies have considered the representativeness and reliability of Wikipedia's scientific sources.…”

COVID-19 research in Wikipedia

“…Wikipedia articles are created, improved and maintained by the efforts of the community of volunteer editors [46, 10], and they are used in a variety of ways by a wide user base [52, 31, 44]. The information Wikipedia contains is generally considered to be of high-quality and up-to-date [46, 24, 18, 29, 45, 5, 53], notwithstanding margins for improvement and the need for constant knowledge maintenance [10, 32, 16].…”

“…Following Wikipedia’s editorial guidelines, the community of editors creates contents often relying on scientific and scholarly literature [40, 19, 6], and therefore Wikipedia can be considered a mainstream gateway to scientific information [30, 20, 32, 50, 34, 44]. Unfortunately, few studies have considered the representativeness and reliability of Wikipedia’s scientific sources.…”

Modeling Popularity and Reliability of Sources in Multilingual Wikipedia

“…One of the works used special identifiers (such as DOI, ISBN) to unify the references and find the similarity of sources between language versions of Wikipedia [12]. Another recent study analyzed engagement with citations in Wikipedia articles and found that references are consulted more commonly when readers cannot find enough information in selected Wikipedia article [13]. There are also works, which showed that a lot of citations in Wikipedia articles refers to scientific publications [12,14], especially if they are open-access [15], wherein Wikipedia authors prefer to put recently published journal articles as a source [16].…”