On Training Knowledge Graph Embedding Models

Abstract

Knowledge graph embedding (KGE) models have become popular means for making discoveries in knowledge graphs (e.g., RDF graphs) in an efficient and scalable manner. The key to success of these models is their ability to learn low-rank vector representations for knowledge graph entities and relations. Despite the rapid development of KGE models, state-of-the-art approaches have mostly focused on new ways to represent embeddings interaction functions (i.e., scoring functions). In this paper, we argue that the choice of other training components such as the loss function, hyperparameters and negative sampling strategies can also have substantial impact on the model efficiency. This area has been rather neglected by previous works so far and our contribution is towards closing this gap by a thorough analysis of possible choices of training loss functions, hyperparameters and negative sampling techniques. We finally investigate the effects of specific choices on the scalability and accuracy of knowledge graph embedding models.

Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework

“…The square error loss, binary cross entropy loss (BCEL), pointwise hinge loss, and logistic loss. a) Square Error Loss: The square error loss function computes the squared difference between the predicted scores and the labels l i ∈ {0, 1} [7]:…”

“…The loss penalizes scores of positive examples which are smaller than λ, but does not impose any restriction on values > λ. Similarly, negative scores larger than −λ contribute to the loss, whereas all values smaller than −λ do not have any loss contribution [7]. Thereby, the model is not encouraged to further optimize triples which are already predicted well enough (according to the margin parameter λ).…”

“…Akrami et al [2], [6] examined the effect of removing defect triples from KGs on the model's performance. Mohamed et al [7] studied the influence of loss functions on the models' performances for a set of KGEMs. Concurrent to our work on this paper, Rufinelli et al performed a benchmarking study in which they investigated five knowledge graph embedding models.…”

Discovering Supply Chain Links with Augmented Intelligence

“…Taken from the link prediction literature, we use "pairwise logistic loss" [47]. Given a positive pair (A, B), we randomly sample a new company C to create a negative pair (A, C).…”

Prediction of Adverse Biological Effects of Chemicals Using Knowledge Graph Embeddings

“…Work on KGE models usually define loss functions specific to the models. However, as show in [48,53] the choice of loss function has a huge impact on model performance. In this work we use four loss functions.…”