Provenance Information in a Collaborative Knowledge Graph: An Evaluation of Wikidata External References

Abstract

Abstract. Wikidata is a collaboratively-edited knowledge graph; it expresses knowledge in the form of subject-property-value triples, which can be enhanced with references to add provenance information. Understanding the quality of Wikidata is key to its widespread adoption as a knowledge resource. We analyse one aspect of Wikidata quality, provenance, in terms of relevance and authoritativeness of its external references. We follow a two-staged approach. First, we perform a crowdsourced evaluation of references. Second, we use the judgements collected in the first stage to train a machine learning model to predict reference quality on a large-scale. The features chosen for the models were related to reference editing and the semantics of the triples they referred to. 61% of the references evaluated were relevant and authoritative. Bad references were often links that changed and either stopped working or pointed to other pages. The machine learning models outperformed the baseline and were able to accurately predict non-relevant and nonauthoritative references. Further work should focus on implementing our approach in Wikidata to help editors find bad references.

A systematic literature review on Wikidata

“…Data quality is also influenced by the relevance and authoritativeness of its external references or sources. Piscopo, Kaffee, Phethean and Simperl (2017) analyzed this particular aspect of Wikidata quality and found external references to be “mostly relevant and authoritative,” and explored models to predict non-relevant or non-authoritative references that could be useful for future applications. Piscopo, Vougiouklis, Kaffee, Phethean, Hare and Simperl (2017) also investigated the relationship between two closely related Wikimedia projects, Wikipedia and Wikidata, from the external references perspective, finding little reuse of references across Wikidata and Wikipedia and less Anglo-American sources in the former (although their references “often point to the same domain” which might be a sign of actual diversification of knowledge across languages).…”

Knowledge Graphs

“…With respect to temporal context [23,44,114,115], Santiago has existed as a city since 1541, flights from Arica to Santiago began in 1956, and so on. With respect to provenance [16,39,103], data about EID15 were taken from-and are thus said to be true with respect to-the Ñam webpage on April 11th, 2020. Other forms of context may also be used and combined, such as to indicate that Arica is a Chilean city (geographic) since 1883 (temporal) per the Treaty of Ancón (provenance).…”

Wikipedia citations: A comprehensive data set of citations with identifiers extracted from English Wikipedia

“…Importantly, such mapping should consider disciplinary differences in citations from Wikipedia, as well as books (5.3 million citations by our estimates) and nonscientific sources such as news outlets and other online media (21.5 million citations), which make up the largest share of Wikipedia citations. Answering these questions is critical to inform the community work on improving Wikipedia by finding and filling knowledge gaps and biases, at the same time guaranteeing the quality and diversity of the sources Wikipedia relies upon (Hube, 2017;Mesgari et al, 2015;Piscopo, Kaffee et al, 2017;Piscopo & Simperl, 2019;Wang & Li, 2020). Link prediction in general, and citation recommendation in particular, have been explored for Wikipedia for some time (Fetahu et al, 2016;Paranjape, West et al, 2016;Wulczyn, West et al, 2016).…”