Applying the information bottleneck to statistical relational learning

Riguzzi, Fabrizio; Mauro, Nicola Di

doi:10.1007/s10994-011-5247-6

Cited by 17 publications

(21 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We implemented EMBLEM in Yap Prolog 4 and we compared it with RIB [31]; CEM, an implementation of EM based on the cplint inference library [27,30]; LeProblog [9,10] and Alchemy [24]. All experiments were performed on Linux machines with an Intel Core 2 Duo E6550 (2333 MHz) processor and 4 GB of RAM.…”

Section: Methodsmentioning

confidence: 99%

“…RIB [31] performs parameter learning using the information bottleneck approach, which is an extension of EM targeted especially towards hidden variables. However, it works best when interpretations have the same Herbrand base, which is not always the case.…”

Section: Related Workmentioning

confidence: 99%

“…Its main characteristic is that it computes the values of expectations using BDDs. EMBLEM is developed for the language of LPADs and tested on the IMDB [21], Cora [34] and UW-CSE [34] datasets and compared with RIB [31], LeProbLog [5], Alchemy [24] and CEM, an implementation of EM based on the cplint interpreter [27].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Expectation maximization over binary decision diagrams for probabilistic logic programs

Bellodi

Riguzzi

2013

IDA

Self Cite

View full text Add to dashboard Cite

Recently much work in Machine Learning has concentrated on using expressive representation languages that combine aspects of logic and probability. A whole field has emerged, called Statistical Relational Learning, rich of successful applications in a variety of domains. In this paper we present a Machine Learning technique targeted to Probabilistic Logic Programs, a family of formalisms where uncertainty is represented using Logic Programming tools. Among various proposals for Probabilistic Logic Programming, the one based on the distribution semantics is gaining popularity and is the basis for languages such as ICL, PRISM, ProbLog and Logic Programs with Annotated Disjunctions. This paper proposes a technique for learning parameters of these languages. Since their equivalent Bayesian networks contain hidden variables, an Expectation Maximization (EM) algorithm is adopted. In order to speed the computation up, expectations are computed directly on the Binary Decision Diagrams that are built for inference. The resulting system, called EMBLEM for "EM over Bdds for probabilistic Logic programs Efficient Mining", has been applied to a number of datasets and showed good performances both in terms of speed and memory usage. In particular its speed allows the execution of a high number of restarts, resulting in good quality of the solutions.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Expectation maximization over binary decision diagrams for probabilistic logic programs

Bellodi

Riguzzi

2013

IDA

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other EM-based approaches for parameter learning include PRISM (Sato and Kameya 2001), LFI-ProbLog (Gutmann et al 2011), ProbLog2 (Fierens et al 2013) and RIB (Riguzzi and Di Mauro 2012).…”

Section: Parameter Learningmentioning

confidence: 99%

Bandit-based Monte-Carlo structure learning of probabilistic logic programs

2015

Self Cite

View full text Add to dashboard Cite

Probabilistic logic programming can be used to model domains with complex and uncertain relationships among entities. While the problem of learning the parameters of such programs has been considered by various authors, the problem of learning the structure is yet to be explored in depth. In this work we present an approximate search method based on a one-player game approach, called LEMUR. It sees the problem of learning the structure of a probabilistic logic program as a multi-armed bandit problem, relying on the Monte-Carlo tree search UCT algorithm that combines the precision of tree search with the generality of random sampling. LEMUR works by modifying the UCT algorithm in a fashion similar to FUSE, that considers a finite unknown horizon and deals with the problem of having a huge branching factor. The proposed system has been tested on various real-world datasets and has shown good performance with respect to other state of the art statistical relational learning approaches in terms of classification abilities.

show abstract

“…Formalisms falling into this category are probabilistic horn abduction (PHA) (Poole, 1993), probabilistic logic programming (PLP) (Ng and Subrahmanian, 1992), relational Bayesian networks (RBNs) (Jaeger, 1997), Bayesian logic programming (BLP) (Kersting et al, 2000), stochastic logic programmes (SLPs) (Muggleton, 1996), PRISM (Sato and Kameya, 1997), CLP(BN) (Costa and Cussens, 2003), ProbLog , and logic programmes with annotated disjunctions (LPADs) (Vennekens et al, 2004). Since some of these languages can be translated into Bayesian networks, when the networks contain hidden variables, learning the parameters of these languages requires the use of techniques for learning from incomplete data such as the expectation maximisation (EM) algorithm (Dempster et al, 1977) or the recent relational information bottleneck (RIB) framework (Riguzzi and Di Mauro, 2012). In the indirect approach, conversely, formulae are not explicitly associated to their probability, and the probability of a possible world is defined in terms of its features by means of an associated real-valued parameter.…”

Section: Statistical Relational Learningmentioning

confidence: 99%

Social networks and statistical relational learning: a survey

Esposito

Ferilli

Basile

et al. 2012

IJSNM

View full text Add to dashboard Cite

One of the most appreciated functionality of computers nowadays is their being a means for communication and information sharing among people. With the spread of the Internet, several complex interactions have taken place among people, giving rise to huge Information Networks based on these interactions. Social Networks potentially represent an invaluable source of information that can be exploited for scientific and commercial purposes. On the other hand, due to their distinguishing peculiarities (huge size and inherent relational setting) with respect to all previous information extraction tasks faced in Computer Science, they require new techniques to gather this information. Social Network Mining (SNM) is the corresponding research area, aimed at extracting information about the network objects and behavior that cannot be obtained based on the explicit/implicit description of the objects alone, ignoring their explicit/implicit relationships. Statistical Relational Learning (SRL) is a very promising approach to SNM, since it combines expressive representation formalisms, able to model complex relational networks, with statistical methods able to handle uncertainty about objects and relations. This paper is a survey of some SRL formalisms and techniques adopted to solve some SNM tasks

show abstract

Applying the information bottleneck to statistical relational learning

Cited by 17 publications

References 31 publications

Expectation maximization over binary decision diagrams for probabilistic logic programs

Expectation maximization over binary decision diagrams for probabilistic logic programs

Bandit-based Monte-Carlo structure learning of probabilistic logic programs

Social networks and statistical relational learning: a survey

Contact Info

Product

Resources

About