Incorporating Literals into Knowledge Graph Embeddings

Abstract

Knowledge graphs are composed of different elements: entity nodes, relation edges, and literal nodes. Each literal node contains an entity's attribute value (e.g. the height of an entity of type person) and thereby encodes information which in general cannot be represented by relations between entities alone. However, most of the existing embedding-or latent-feature-based methods for knowledge graph analysis only consider entity nodes and relation edges, and thus do not take the information provided by literals into account. In this paper, we extend existing latent feature methods for link prediction by a simple portable module for incorporating literals, which we name Lit-eralE. Unlike in concurrent methods where literals are incorporated by adding a literal-dependent term to the output of the scoring function and thus only indirectly affect the entity embeddings, LiteralE directly enriches these embeddings with information from literals via a learnable parametrized function. This function can be easily integrated into the scoring function of existing methods and learned along with the entity embeddings in an end-to-end manner. In an extensive empirical study over three datasets, we evaluate LiteralE-extended versions of various state-of-the-art latent feature methods for link prediction and demonstrate that LiteralE presents an effective way to improve their performance. For these experiments, we augmented standard datasets with their literals, which we publicly provide as testbeds for further research. Moreover, we show that LiteralE leads to an qualitative improvement of the embeddings and that it can be easily extended to handle literals from different modalities.

Message Passing for Hyper-Relational Knowledge Graphs

“…Literals (e.g. numeric values, date-time instances or other strings, etc) in KGs are conventionally ignored (Rosso et al, 2020) by embedding approaches, or are incorporated through specific means (Kristiadi et al, 2019). However, after removing statements with literals, less than 3% of the remaining statements contain any qualifier pairs.…”

Multi-source knowledge fusion: a survey

“…Kristiadi et al [38] considered the semantic information carried by the literal meanings of entity names in knowledge graphs, and proposed a new representation learning mechanism LiteralE (See Fig. 5).…”

Joint Embedding Learning of Educational Knowledge Graphs

“…TEKE [49] defines context vectors of entities and relations, and combines context vectors into traditional models, e.g., TransE, Xie et al [51] and Xu et al [52] encode textual literals by convolutional and recurrent neural networks. LiteralE [24] replaces the original entity embeddings of conventional loss functions with literal-enriched vectors, which are defined by learnable parametrized functions. SimplE [23] incorporates certain types of background knowledge into the model by weight tying.…”

“…How to appropriately incorporate massive literals attached to different entities of educational KGs into embedding representations? We have noticed a model named LiteralE [24] which utilizes literals for embedding learning by a simple portable module. However, LiteralE only considers numerical literals which are apparently a very limited part of literals of educational KGs.…”