Learning compact Markov logic networks with decision trees

Abstract

Statistical-relational learning combines logical syntax with probabilistic methods. Markov Logic Networks (MLNs) are a prominent model class that generalizes both firstorder logic and undirected graphical models (Markov networks). The qualitative component of an MLN is a set of clauses and the quantitative component is a set of clause weights. Generative MLNs model the joint distribution of relationships and attributes. A state-ofthe-art structure learning method is the moralization approach: learn a set of directed Horn clauses, then convert them to conjunctions to obtain MLN clauses. The directed clauses are learned using Bayes net methods. The moralization approach takes advantage of the high-quality inference algorithms for MLNs and their ability to handle cyclic dependencies. A weakness of moralization is that it leads to an unnecessarily large number of clauses. In this paper we show that using decision trees to represent conditional probabilities in the Bayes net is an effective remedy that leads to much more compact MLN structures. In experiments on benchmark datasets, the decision trees reduce the number of clauses in the moralized MLN by a factor of 5-25, depending on the dataset. The accuracy of predictions is competitive with the models obtained by standard moralization, and in many cases superior.

Learning directed relational models with recursive dependencies

“…Our code and datasets are available on the world-wide web (Khosravi et al 2012). We made use of the following existing implementations.…”

Learning Bayes Nets for Relational Data with Link Uncertainty

“…However to our knowledge, no implementations of such structure learning algorithms for directed graphical models are available. Our system builds on the state-of-the-art Bayes net learner for relational data, whose code is available at [6]. Implementations exist for other types of graphical models, specifically Markov random fields (undirected models) [2] and dependency networks (directed edges with cycles allowed) [10].…”

Simple decision forests for multi-relational classification

“…In terms of graphical models, an MLN is a template for a Markov random field, with a log-linear likelihood function that is the weighted sum of counts of features defined by the first-order formulas. A state-of-the-art approach to learning the clauses in an MLN is to first learn a set of decision trees and then convert each branch of each decision tree to an MLN clause [36,33]. The weights of the clauses are obtained from probability estimation trees by using the log-conditional probabilities associated with a leaf [36], and from regression trees by using the regression weights [33].…”

“…A state-of-the-art approach to learning the clauses in an MLN is to first learn a set of decision trees and then convert each branch of each decision tree to an MLN clause [36,33]. The weights of the clauses are obtained from probability estimation trees by using the log-conditional probabilities associated with a leaf [36], and from regression trees by using the regression weights [33]. The example rule from Figure 4 (right) would induce the MLN clause RA(S , P ), TeachingA(P , 1 ), Intelligence(S , 2 ) : w = ln(100 %).…”

“…For general discussion and details on MLNs and the conversion from decision forests to MLNs please see [36,33]. The conversion to Markov Logic Networks provides a probabilistic foundation for our classification approach in terms of log-linear models, and o↵ers a principled approach to collective classification with the cross-table Naive Bayes assumption (see Sec.…”