Lifted Rule Injection for Relation Embeddings

Abstract

Methods based on representation learning currently hold the state-of-the-art in many natural language processing and knowledge base inference tasks. Yet, a major challenge is how to efficiently incorporate commonsense knowledge into such models. A recent approach regularizes relation and entity representations by propositionalization of first-order logic rules. However, propositionalization does not scale beyond domains with only few entities and rules. In this paper we present a highly efficient method for incorporating implication rules into distributed representations for automated knowledge base construction. We map entity-tuple embeddings into an approximately Boolean space and encourage a partial ordering over relation embeddings based on implication rules mined from WordNet. Surprisingly, we find that the strong restriction of the entity-tuple embedding space does not hurt the expressiveness of the model and even acts as a regularizer that improves generalization. By incorporating few commonsense rules, we achieve an increase of 2 percentage points mean average precision over a matrix factorization baseline, while observing a negligible increase in runtime.

Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning

“…(?, r , o) with right answer s, we first fit subject entity position with each entity e ∈ E and thus get a set of triples T sub jectpr edict ion = {(e, r , o)|e ∈ E}. Then we calculate the score for each triple in T sub jectpr edict ion according to Equation (6) and rank their scores in descending order. Thus the entity e in (e, r , o) with a higher rank is a more possible prediction.…”

“…[21] adds constrains for relation embeddings participating in path rules. [6] maps entity-tuple embedding into an approximately Boolean space and encourages a partial ordering over relation embedding based on implication rules. [8] adds approximate entailment constraints on relation representations.…”

A Boxology of Design Patterns forHybrid Learningand Reasoning Systems

“…An example of this are the Logic Tensor Networks in [46], where the authors show that encoding prior knowledge in symbolic form allows for better learning results on fewer training data, as well as more robustness against noise. A similar example is given in [47], where knowledge graphs are successfully used as priors in a scene description task, and in [48] where logical rules are used as background knowledge for a gradient descent learning task in a high-dimensional real-valued vector space.…”

MHCSeqNet: A deep neural network model for universal MHC binding prediction

“…The Skip-Gram Model (Demeester et al, 2016) was used to train the continuous vectors by treating each set of 1 or 3 consecutive amino acids (1-gram or 3-gram) as a unit. The choice of the 3-gram model, also called ProtVec, was selected according to an earlier study (Asgari and Mofrad, 2015).…”