Learning Predictive Categories Using Lifted Relational Neural Networks

Abstract

Abstract. Lifted relational neural networks (LRNNs) are a flexible neuralsymbolic framework based on the idea of lifted modelling. In this paper we show how LRNNs can be easily used to specify declaratively and solve learning problems in which latent categories of entities, properties and relations need to be jointly induced.

Ontology Completion Using Graph Convolutional Networks

“…Here e h and e t are the embeddings of the entities h and t, and R r is a diagonal matrix representing the relation r. The ComplEx model [43] is an extension of [48] in the complex space. A different strategy consists in learning latent soft clusters of predicates to predict missing facts in relational data, for example by using Markov logic network [24] or by applying neural network models [36,40]. Several rule-based approaches have also been proposed, where observed regularities in the given knowledge graph are summarized as a weighted set of rules, which is then used to derive plausible missing facts.…”

“…For example, [11] proposed a system inspired by inductive logic programming, while [44] introduced statistical schema induction to mine association rule from RDF data and then generate ontologies. More recently, [40] used so-called Lifted Relational Neural Networks to learn rules in an implicit way. In [31], meta-rules were found automatically by meta-interpretive learning.…”

“…Similar to knowledge graphs, existing ontologies are often incomplete, hence there is a need for methods that can automatically predict plausible missing rules for a given ontology. For some ontologies, where we have a large database of facts (often called an ABox), plausible rules can be learned similarly to how rules are learned in inductive logic programming and statistical relational learning [11,31,40,44]. However, for many commonly used ontologies, such a database of facts is not available.…”

Stacked Structure Learning for Lifted Relational Neural Networks

“…The structure learning algorithm will create LRNNs having a generic "stacked" structure which we now describe. First, there are rules that define d new predicates, representing soft clusters [17] of unary predicates from the dataset. These can be thought of as the first layer of the LRNN, where the weighted facts from the dataset comprise the zeroth layer.…”

“…In Figure 5, we plot the evolution of these embeddings as new rules are being added by the structure learning algorithm. The left panel of Figure 5 displays the evolution of the embeddings of atom types after these have been pre-trained using an unsupervised method which was originally used for statistical predicate invention in [17]. The right panel of the same figure displays the evolution of the embeddings when starting from random initialization without any unsupervised pre-training.…”