A Logic-Driven Framework for Consistency of Neural Models

Li, Tao; Gupta, Vivek; Mehta, Maitrey; Srikumar, Vivek

doi:10.18653/v1/d19-1405

Cited by 61 publications

(50 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subsequently, our method uses symbolic logic to incorporate consistency regularization for additional supervision signal beyond inductive bias given by data augmentation. Our method generalizes previous consistency-promoting methods for NLI tasks (Minervini and Riedel, 2018;Li et al, 2019) to adapt to substantially different question formats.…”

Section: Introductionmentioning

confidence: 86%

“…Minervini and Riedel (2018) present model-dependent first-order logic guided adversarial example generation and regularization. Li et al (2019) introduce consistency-based regularization incorporating the first-order logic rules. Previous approach is modeldependent or relies on NLI-specific rules, while our method is model-agnostic and is more generally applicable by combining it with data augmentation.…”

Section: Related Workmentioning

confidence: 99%

“…Importantly, state-of-the-art models show inconsistent comparison predictions as shown in Figure 1. Improving the consistency of predictions has been previously studied in natural language inference (NLI) tasks (Minervini and Riedel, 2018;Li et al, 2019), but has not been addressed in QA.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

Asai¹,

Hajishirzi²

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Many natural language questions require qualitative, quantitative or logical comparisons between two entities or events. This paper addresses the problem of improving the accuracy and consistency of responses to comparison questions by integrating logic rules and neural models. Our method leverages logical and linguistic knowledge to augment labeled training data and then uses a consistency-based regularizer to train the model. Improving the global consistency of predictions, our approach achieves large improvements over previous methods in a variety of question answering (QA) tasks including multiple-choice qualitative reasoning, cause-effect reasoning, and extractive machine reading comprehension. In particular, our method significantly improves the performance of RoBERTa-based models by 1-5% across datasets. We advance state of the art by around 5-8% on WIQA and QuaRel and reduce consistency violations by 58% on HotpotQA. We further demonstrate that our approach can learn effectively from limited data. 1

show abstract

Section: Introductionmentioning

confidence: 86%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

Asai¹,

Hajishirzi²

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…The goal of learning is to let the model capture the data annotation, meanwhile regularizing the model towards consistency on logic constraints. Inspired by the logicdriven framework for consistency of neural models (Li et al, 2019), we specify three types of consistency requirements, i.e. annotation consistency, symmetry consistency and conjunction consistency.…”

Section: Joint Constrained Learningmentioning

confidence: 99%

Joint Constrained Learning for Event-Event Relation Extraction

Wang

Chen

Zhang

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other.In this process, one can induce event complexes that organize multi-granular events with temporal order and membership relations interweaving among them. Due to the lack of jointly labeled data for these relational phenomena and the restriction on the structures they articulate, we propose a joint constrained learning framework for modeling event-event relations. Specifically, the framework enforces logical constraints within and across multiple temporal and subevent relations by converting these constraints into differentiable learning objectives. We show that our joint constrained learning approach effectively compensates for the lack of jointly labeled data, and outperforms SOTA methods on benchmarks for both temporal relation extraction and event hierarchy construction, replacing a commonly used but more expensive global inference process. We also present a promising case study showing the effectiveness of our approach in inducing event complexes on an external corpus. 1 * This work was done when the author was visiting the University of Pennsylvania.

show abstract

“…State-of-the-art BERT (Devlin et al, 2019) representations have boosted performance in a wide variety of NLP tasks. The rising interest is on frameworks that combine neural network-driven representations with logic representations to reason about language and predict correct outputs for tasks such as natural language inference (NLI) (Li et al, 2019). Logic puzzle solving is a task that is considered in this direction, as well.…”

Section: Introductionmentioning

confidence: 99%

LGPSolver - Solving Logic Grid Puzzles Automatically

Jabrayilzade

Tekir

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Logic grid puzzle (LGP) is a type of word problem where the task is to solve a problem in logic. Constraints for the problem are given in the form of textual clues. Once these clues are transformed into formal logic, a deductive reasoning process provides the solution.Solving logic grid puzzles in a fully automatic manner has been a challenge since a precise understanding of clues is necessary to develop the corresponding formal logic representation. To meet this challenge, we propose a solution that uses a DistilBERT-based classifier to classify a clue into one of the predefined predicate types for logic grid puzzles. Another novelty of the proposed solution is the recognition of comparison structures in clues. By collecting comparative adjectives from existing dictionaries and utilizing a semantic framework to catch comparative quantifiers, the semantics of clues concerning comparison structures are better understood, ensuring conversion to correct logic representation. Our approach solves logic grid puzzles in a fully automated manner with 100% accuracy on the given puzzle datasets and outperforms state-of-the-art solutions by a large margin.

show abstract

A Logic-Driven Framework for Consistency of Neural Models

Cited by 61 publications

References 31 publications

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

Joint Constrained Learning for Event-Event Relation Extraction

LGPSolver - Solving Logic Grid Puzzles Automatically

Contact Info

Product

Resources

About