End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion

Abstract

Knowledge graph embedding has been an active research topic for knowledge base completion, with progressive improvement from the initial TransE, TransH, DistMult et al to the current state-of-the-art ConvE. ConvE uses 2D convolution over embeddings and multiple layers of nonlinear features to model knowledge graphs. The model can be efficiently trained and scalable to large knowledge graphs. However, there is no structure enforcement in the embedding space of ConvE. The recent graph convolutional network (GCN) provides another way of learning graph node embedding by successfully utilizing graph connectivity structure. In this work, we propose a novel end-to-end Structure-Aware Convolutional Network (SACN) that takes the benefit of GCN and ConvE together. SACN consists of an encoder of a weighted graph convolutional network (WGCN), and a decoder of a convolutional network called Conv-TransE. WGCN utilizes knowledge graph node structure, node attributes and edge relation types. It has learnable weights that adapt the amount of information from neighbors used in local aggregation, leading to more accurate embeddings of graph nodes. Node attributes in the graph are represented as additional nodes in the WGCN. The decoder Conv-TransE enables the state-of-the-art ConvE to be translational between entities and relations while keeps the same link prediction performance as ConvE. We demonstrate the effectiveness of the proposed SACN on standard FB15k-237 and WN18RR datasets, and it gives about 10% relative improvement over the state-of-theart ConvE in terms of HITS@1, HITS@3 and HITS@10.

Mining Implicit Entity Preference from User-Item Interaction Data for Knowledge Graph Completion via Adversarial Learning

“…We either follow the reported optimal parameters or optimize each model separately using validation set. Following [6,25], we equip semantic-matching based methods with 1 − N scoring strategy, including DistMult that previously adopted a simple binary entropy cross loss. To avoid overfitting, we adopt early stopping by evaluating MRR on the validation set every 20 epochs.…”

“…More recently, Graph Neural Network (GNN) [15,32] has received much attention as an effective technique to learn node embeddings over graph-structured data. Several studies try to utilize GNN to capture semantic relations on the KG, such as relational convolution [24] and structural convolution [25]. However, these methods mainly focus on modeling KG graph structure, which cannot effectively integrate user interaction data.…”

Knowledge Graph Completion: A Review

“…First of all, the graph is the most representative of local connection structure. Second, compared with the traditional spectrum diagram theory [96], shared weight reduces the computational cost. Finally, multilayer structure is the key to dealing with hierarchical models.…”

Rotate3D: Representing Relations as Rotations in Three-Dimensional Space for Knowledge Graph Embedding

“…ConvE [4] uses a multi-layer convolutional architecture to define score functions. SACN [14] leverages graph structure by introducing a weighted graph convolutional network as the encoder to improve ConvE. InteractE [20] uses three operations: feature permutation, checkered feature reshaping, and circular convolution to augment the expressive power of ConvE.…”