Efficient EM Learning with Tabulation for Parameterized Logic Programs

Kameya, Yoshitaka; Sato, Taisuke

doi:10.1007/3-540-44957-4_18

Cited by 26 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…R(G) denotes the set of all refutations of G; R(G) can be empty, finite or infinite. D(G) denotes the set of all derivations of G. Kameya and Sato (2000) note that "there are two different basic attitudes towards the use of probability in logic or logic programming". These are the constraint approach where the probability that a logical formula is true is constrained to lie in some region and the distribution approach where a specific probability distribution is defined which gives the probability that each logical formula is true.…”

Section: Logic Programming Definitionsmentioning

confidence: 99%

“…The work in Kameya and Sato (2000) on applying EM to models described in the PRISM (programming in statistical modelling) language addresses the problem by using tabulation to increase efficiency. Tabular approaches can take advantage of the shared common structure of different refutations and avoid repeated computations.…”

Section: Using Fammentioning

confidence: 99%

“…PRISM models define distributions where every ground atom represents a binary random variable with possible values true and false. SLPs represent a different type of distribution, so it is difficult to see how the EM approach in Kameya and Sato (2000) could be directly applied to SLP parameter estimation. Nonetheless there are interesting connections and a tabular approach seems the sensible way to compute exact expectations in SLPs.…”

Section: Using Fammentioning

confidence: 99%

See 2 more Smart Citations

Untitled

Cussens

2001

Machine Learning

View full text Add to dashboard Cite

Abstract. Stochastic logic programs (SLPs) are logic programs with parameterised clauses which define a loglinear distribution over refutations of goals. The log-linear distribution provides, by marginalisation, a distribution over variable bindings, allowing SLPs to compactly represent quite complex distributions.We analyse the fundamental statistical properties of SLPs addressing issues concerning infinite derivations, 'unnormalised' SLPs and impure SLPs. After detailing existing approaches to parameter estimation for log-linear models and their application to SLPs, we present a new algorithm called failure-adjusted maximisation (FAM). FAM is an instance of the EM algorithm that applies specifically to normalised SLPs and provides a closed-form for computing parameter updates within an iterative maximisation approach. We empirically show that FAM works on some small examples and discuss methods for applying it to bigger problems.

show abstract

Section: Logic Programming Definitionsmentioning

confidence: 99%

Section: Using Fammentioning

confidence: 99%

Section: Using Fammentioning

confidence: 99%

See 1 more Smart Citation

Untitled

Cussens

2001

Machine Learning

View full text Add to dashboard Cite

show abstract

“…whatever is unobserved in the world is considered to be false. Research with missing data in SRL has mainly focused on learning the parameters where algorithms based on classical EM (Dempster et al 1977) have been developed (Natarajan et al 2008;Jaeger 2007;Xiang and Neville 2008;Kameya and Sato 2000;Gutmann et al 2011;Riguzzi 2013, 2012). There has also been some work on learning structure of SRL models from hidden data (Li and Zhou 2007;Kersting and Raiko 2005).…”

Section: Introductionmentioning

confidence: 99%

Gradient-based boosting for statistical relational learning: the Markov logic network and missing data cases

et al. 2015

View full text Add to dashboard Cite

Recent years have seen a surge of interest in Statistical Relational Learning (SRL) models that combine logic with probabilities. One prominent and highly expressive SRL model is Markov Logic Networks (MLNs), but the expressivity comes at the cost of learning complexity. Most of the current methods for learning MLN structure follow a two-step approach where first they search through the space of possible clauses (i.e. structures) and then learn weights via gradient descent for these clauses. We present a functional-gradient boosting algorithm to learn both the weights (in closed form) and the structure of the MLN simultaneously. Moreover most of the learning approaches for SRL apply the closed-world assumption, i.e., whatever is not observed is assumed to be false in the world. We attempt to open this assumption. We extend our algorithm for MLN structure learning to handle missing data by using an EM-based approach and show this algorithm can also be used to learn Relational Dependency Networks and relational policies. Our results in many domains demonstrate that our approach can effectively learn MLNs even in the presence of missing data.

show abstract

“…The gEM algorithm was originally proposed by Kameya and Sato (2000) for a probabilistic logic programming language called PRISM (Sato and Kameya 1997). PRISM's semantic basis is the distribution semantics (Sato 1995), which is a probabilistic extension of the least model semantics in logic programs.…”

mentioning

confidence: 99%

Using WFSTs for Efficient EM Learning of Probabilistic CFGs and Their Extensions

Kameya

Mori

Sato

2014

Journal of Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Probabilistic context-free grammars (PCFGs) are a widely known class of probabilistic language models. The Inside-Outside (I-O) algorithm is well known as an efficient EM algorithm tailored for PCFGs. Although the algorithm requires inexpensive linguistic resources, there remains a problem in its efficiency. This paper presents an efficient method for training PCFG parameters in which the parser is separated from the EM algorithm, assuming that the underlying CFG is given. A new EM algorithm exploits the compactness of well-formed substring tables (WFSTs) generated by the parser. Our proposal is general in that the input grammar need not take Chomsky normal form (CNF) while it is equivalent to the I-O algorithm in the CNF case. In addition, we propose a polynomial-time EM algorithm for CFGs with contextsensitive probabilities, and report experimental results with the ATR dialogue corpus and a hand-crafted Japanese grammar.

show abstract

Efficient EM Learning with Tabulation for Parameterized Logic Programs

Cited by 26 publications

References 12 publications

Untitled

Untitled

Gradient-based boosting for statistical relational learning: the Markov logic network and missing data cases

Using WFSTs for Efficient EM Learning of Probabilistic CFGs and Their Extensions

Contact Info

Product

Resources

About