Learning Continuous Semantic Representations of Symbolic Expressions

Allamanis, Miltiadis; Chanthirasegaran, Pankajan; Kohli, Pushmeet; Sutton, Charles

doi:10.48550/arxiv.1611.01423

Cited by 5 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related work Data-driven models have proven successful in various entailment recognition tasks (Baroni et al, 2012;Socher et al, 2012;Rocktäschel et al, 2014;Bowman et al, 2015b;Rocktäschel et al, 2015). The data sets used in research on this topic tend to be either fully formal, focusing on logic instead of natural language Allamanis et al, 2016), or fully natural, as is the case for manually annotated data sets of English sentence pairs such as SICK (Marelli et al, 2014) or SNLI (Bowman et al, 2015a). Moreover, entailment recognition models are often endowed with functionality reflecting pre-established linguistic or semantic regularities of the data (Bankova et al, 2016;Serafini and Garcez, 2016;Sadrzadeh et al, 2018).…”

Section: Introduction and Related Workmentioning

confidence: 99%

Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization

Mul,

Zuidema

2019

Preprint

View full text Add to dashboard Cite

Can neural nets learn logic? We approach this classic question with current methods, and demonstrate that recurrent neural networks can learn to recognize first order logical entailment relations between expressions. We define an artificial language in first-order predicate logic, generate a large dataset of sample 'sentences', and use an automatic theorem prover to infer the relation between random pairs of such sentences. We describe a Siamese neural architecture trained to predict the logical relation, and experiment with recurrent and recursive networks. Siamese Recurrent Networks are surprisingly successful at the entailment recognition task, reaching near perfect performance on novel sentences (consisting of known words), and even outperforming recursive networks. We report a series of experiments to test the ability of the models to perform compositional generalization. In particular, we study how they deal with sentences of unseen length, and sentences containing unseen words. We show that set-ups using LSTMs and GRUs obtain high scores on these tests, demonstrating a form of compositionality.Preprint. Under review.

show abstract

Section: Introduction and Related Workmentioning

confidence: 99%

Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization

Mul,

Zuidema

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Second, their dataset consists of matrix expressions containing at most one variable, while our formulas contain many variables. Allamanis et al (2016) use a recursive neural network to learn whether two expressions are equivalent. They tested on two datasets: propositional logic and polynomials.…”

Section: Resultsmentioning

confidence: 99%

“…The fourth and fifth encoding benchmarks are (tree) recursive neural networks (Tai et al, 2015;Le & Zuidema, 2015;Zhu et al, 2015;Allamanis et al, 2016), also known as TreeRNNs. These recursively encode the logical expression using the parse structure ‡ , where leaf nodes of the tree (propositional variables) are embedded as learnable vectors, and each logical operator then combines one or more of these embedded values to produce a new embedding.…”

Section: Encoder Benchmarksmentioning

confidence: 99%

“…For example, the expression (¬a) ∨ b is parsed as the tree with leaves a and b, a unary node ¬ (with input the embedding of a), and a binary node ∨ (with inputs the embeddings of ¬a and b). Following Allamanis et al (2016), the fourth encoding benchmark is a simple TreeRNN (TreeNet Encoders), where each operator 'op' concatenates its inputs to a vector x, and produces the output…”

Section: Encoder Benchmarksmentioning

confidence: 99%

See 1 more Smart Citation

Can Neural Networks Understand Logical Entailment?

Evans¹,

Saxton²,

David³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We use this task to compare a series of architectures which are ubiquitous in the sequence-processing literature, in addition to a new model class-PossibleWorldNets-which computes entailment as a "convolution over possible worlds". Results show that convolutional networks present the wrong inductive bias for this class of problems relative to LSTM RNNs, treestructured neural networks outperform LSTM RNNs due to their enhanced ability to exploit the syntax of logic, and PossibleWorldNets outperform all benchmarks.

show abstract

“…There are various papers with datasets with a discrete reasoning nature. Kaiser & Sutskever (2015) use an adapted convolutional architecture to solve addition and multiplication with good generalization; Allamanis et al (2016) and Evans et al (2018) use tree networks to predict polynomial or logical equivalence or logical entailment; Selsam et al (2018) uses message passing networks with a bipartite graph structure to decide satisfiability in formulas in conjunctive normal form, and so on. The difference between those problems and the dataset in this paper is that the former all have a single well-defined input structure that can be easily mapped into narrow architectures suited to the problem structure, avoiding the need for general reasoning skills like parsing or generic working memory.…”

Section: Related Workmentioning

confidence: 99%

Analysing Mathematical Reasoning Abilities of Neural Models

Saxton¹,

Grefenstette²,

Hill³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Mathematical reasoning-a core ability within human intelligence-presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.

show abstract

Learning Continuous Semantic Representations of Symbolic Expressions

Cited by 5 publications

References 0 publications

Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization

Siamese recurrent networks learn first-order logic reasoning and exhibit zero-shot compositional generalization

Can Neural Networks Understand Logical Entailment?

Analysing Mathematical Reasoning Abilities of Neural Models

Contact Info

Product

Resources

About