Link Prediction with Attention Applied on Multiple Knowledge Graph Embedding Models

Abstract

Predicting missing links between entities in a knowledge graph is a fundamental task to deal with the incompleteness of data on the Web. Knowledge graph embeddings map nodes into a vector space to predict new links, scoring them according to geometric criteria. Relations in the graph may follow patterns that can be learned, e.g., some relations might be symmetric and others might be hierarchical. However, the learning capability of different embedding models varies for each pattern and, so far, no single model can learn all patterns equally well. In this paper, we combine the query representations from several models in a unified one to incorporate patterns that are independently captured by each model. Our combination uses attention to select the most suitable model to answer each query. The models are also mapped onto a non-Euclidean manifold, the Poincaré ball, to capture structural patterns, such as hierarchies, besides relational patterns, such as symmetry. We prove that our combination provides a higher expressiveness and inference power than each model on its own. As a result, the combined model can learn relational and structural patterns. We conduct extensive experimental analysis with various link prediction benchmarks showing that the combined model outperforms individual models, including state-of-the-art approaches.

Vaccine Coverage and Effectiveness in a School-Based Varicella Outbreak in Jinan Prefecture, Shandong Province

“…Knowledge graph (KG) is a structured representation of knowledge, organizing information into entities and relationships to facilitate understanding and reasoning. KG embedding maps entities and relationships into vector space, effectively encoding structural and semantic information for link prediction [3] and other downstream applications [4]. Federated KG embedding learning [5] [6] [7] extends this approach to decentralized environments, enabling collaborative model training across distributed data sources while preserving data privacy.…”