Crowdsourcing Taxonomies

Abstract

Taxonomies are great for organizing and searching web content. As such, many popular classes of web applications, utilize them. However, their manual generation and maintenance by experts is a timecostly procedure, resulting in static taxonomies. On the other hand, mining and statistical approaches may produce low quality taxonomies. We thus propose a drastically new approach, based on the proven, increased human involvement and desire to tag/annotate web content. We define the required input from humans in the form of explicit structural, e.g., supertype-subtype relationships between concepts. Hence we harvest, via common annotation practices, the collective wisdom of users with respect to the (categorization of) web content they share and access. We further define the principles upon which crowdsourced taxonomy construction algorithms should be based. The resulting problem is NP-Hard. We thus provide and analyze heuristic algorithms that aggregate human input and resolve conflicts. We evaluate our approach with synthetic and realworld crowdsourcing experiments and on a real-world taxonomy.

Provenance Information in a Collaborative Knowledge Graph: An Evaluation of Wikidata External References

“…Conversely, the classification tasks T3.A and T3.B were harder. This may be due to the classification system used appearing unclear for workers, or clashing with their prior knowledge, leading to erroneous responses, similar to what has been noted before in taxonomy creation tasks [14]. Nevertheless, the judgements collected in T3.B largely confirmed T3.A.…”

Linking and clustering artworks using social tags: Revitalizing crowd‐sourced information on cultural collections

“…Low quality of the tags and their lack of structure have been pointed out as the main problems of the social tagging system (Visser, 2013). In addition, as the number of social tags grows rapidly, it is easy for users to get lost, leading to low findability (Karampinas & Triantafillou, 2012;Quintarelli, Resmini, & Rosati, 2007;Stock, 2007). This undermines the role of social tags as a meaningful bridge (Smith, 2006).…”

CrowdTC: Crowdsourced Taxonomy Construction

“…In [12], [21], the crowd is used for categorizing items for taxonomy construction. In [7], each "isA" relationship is voted by the crowd and take it as the input for taxonomy induction. S. K. Kondreddi [13] proposes a hybrid approach that combines information extraction technique with human computation for knowledge acquisition, in which the crowd are asked to compile relationships between entities.…”

“…Existing approaches for taxonomy construction can be categorized mainly into two mainstreams: machine-based automatic construction [5], [6], [4] and human-based manual construction [7], [3]. Each approach has its own advantages and disadvantages, in terms of accuracy, efficiency and cost, respectively.…”