Taxonomy enrichment with text and graph vector representations

Abstract

Knowledge graphs such as DBpedia, Freebase or Wikidata always contain a taxonomic backbone that allows the arrangement and structuring of various concepts in accordance with hypo-hypernym (“class-subclass”) relationship. With the rapid growth of lexical resources for specific domains, the problem of automatic extension of the existing knowledge bases with new words is becoming more and more widespread. In this paper, we address the problem of taxonomy enrichment which aims at adding new words to the existing taxonomy. We present a new method which allows achieving high results on this task with little effort. It uses the resources which exist for the majority of languages, making the method universal. We extend our method by incorporating deep representations of graph structures like node2vec, Poincaré embeddings, GCN etc. that have recently demonstrated promising results on various NLP tasks. Furthermore, combining these representations with word embeddings allows us to beat the state of the art. We conduct a comprehensive study of the existing approaches to taxonomy enrichment based on word and graph vector representations and their fusion approaches. We also explore the ways of using deep learning architectures to extend taxonomic backbones of knowledge graphs. We create a number of datasets for taxonomy extension for English and Russian. We achieve state-of-the-art results across different datasets and provide an in-depth error analysis of mistakes.

Predicting Terms in IS-A Relations with Pre-trained Transformers

“…There exist several recent papers on Taxonomy Enrichment that make use of word vector representations and/or large pre-trained language models. For instance, (Nikishina et al, 2022a) present an approach applying numerous of text and graph embeddings as well as their combinations; (Takeoka et al, 2021) solves the same problem, but for the low-resource scenario using BERT-based classifier, while Roller et al (2018) revise Hearst Patterns for the task. Cho et al (2020), view taxonomy enrichment as a sequence-to-sequence problem and the authors train an LSTM model on the WordNet data.…”

Editorial of the Special Issue on Deep Learning and Knowledge Graphs

“…The sixth paper, "Taxonomy Enrichment with Text and Graph Vector Representations" [16], by Irina Nikishina, Mikhail Tikhomirov, Varvara Logacheva, Yuriy Nazarov, Alexander Panchenko, and Natalia Loukachevitch, targets the problem of taxonomy enrichment which aims at adding new words to the existing taxonomy. This paper provides a comprehensive study of the existing approaches to taxonomy enrichment based on word and graph vector representations.…”