Testing the validity of Wikipedia categories for subject matter labelling of open-domain corpus data

Abstract

The Wikipedia category system was designed to enable browsing and navigation of Wikipedia. It is also a useful resource for knowledge organisation and document indexing, especially using automatic approaches. However, it has received little attention as a resource for manual indexing. In this article, a hierarchical taxonomy of three-level depth is extracted from the Wikipedia category system. The resulting taxonomy is explored as a lightweight alternative to expert-created knowledge organisation systems (e.g. library classification systems) for the manual labelling of open-domain text corpora. Combining quantitative and qualitative data from a crowd-based text labelling study, the validity of the taxonomy is tested and the results quantified in terms of interrater agreement. While the usefulness of the Wikipedia category system for automatic document indexing is documented in the pertinent literature, our results suggest that at least the taxonomy we derived from it is not a valid instrument for manual subject matter labelling of open-domain text corpora.

Assessing knowledge organization systems from a gender perspective: Wikipedia taxonomy and Wikidata ontologies

“…Bolotnikova et al (2011) propose practical methods for ontology evaluation, especially in automated contexts. Aghaebrahimian et al (2022) explore the validity of Wikipedia categories for topic labeling, further contributing to the development of our evaluation criteria.…”

Language-agnostic Topic Classification for Wikipedia

“…9 Not all language communities of Wikipedia, however, use the same set of high-level categories or label articles with categories to the same extent [17]. The category network also is messy, requiring careful rules and pruning to avoid loops when linking a given category to its high-level topic [1,17,22]. WikiProjects represent another annotation system where groups of editors interested in a specific topic area-e.g., Medicine [12]tag articles that are relevant so that they may be evaluated and improved.…”