Learning Translation-Based Knowledge Graph Embeddings by N-Pair Translation Loss

Abstract

Translation-based knowledge graph embeddings learn vector representations of entities and relations by treating relations as translation operators over the entities in an embedding space. Since the translation is represented through a score function, translation-based embeddings are trained in general by minimizing a margin-based ranking loss, which assigns a low score to positive triples and a high score to negative triples. However, this type of embedding suffers from slow convergence and poor local optima because the loss adopts only one pair of a positive and a negative triple at a single update of learning parameters. Therefore, this paper proposes the N-pair translation loss that considers multiple negative triples at one update. The N-pair translation loss employs multiple negative triples as well as one positive triple and allows the positive triple to be compared against the multiple negative triples at each parameter update. As a result, it becomes possible to obtain better vector representations rapidly. The experimental results on link prediction prove that the proposed loss helps to quickly converge toward good optima at the early stage of training.

Modelling Multi-relations for Convolutional-based Knowledge Graph Embedding

“…To better model complex relations, some models [4,13] allow the entity or the relation to have distinct representations. However, Song et al [15] argue that translational distance models suffer from slow convergence and poor local optimality. Moreover, these shallow models are limited to their expressiveness [16].…”

TransET: Knowledge Graph Embedding with Entity Types

“…A similar approach is TranSparse [5], which takes the complexity of the relationship into account in the design of the relational mapping matrix. Song et al [15] argue that the loss of the existing translationbased embedding model uses only a pair of positive and negative triples in one update of the learning parameters, which suffers from slow convergence and poor local optimality. Therefore, they proposed the N-pair translation loss that considers multiple negative triples at one update.…”

Drug Repurposing Using Knowledge Graph Embeddings with a Focus on Vector-Borne Diseases: A Model Comparison

“…Differences between the various embedding algorithms are related to three aspects: (i) how they represent entities and relations, (ii) how they define the scoring function, and (iii) how they optimize the ranking criterion that maximizes the global plausibility of the existing triples [16]. Some of the embedding models that show state-of-the-art performance in knowledge graph completion relate to translation-based models that treat relations as translation operators over the entities in an embedding space [42] (e.g., TransE, TransR, TransH, UM). Alternatively, semantic matching models that use semantic similarity between entities and relations in the embedding space are commonly used for the task (e.g., DistMult, RESCAL, and ERMLP).…”