Explain Yourself! Leveraging Language Models for Commonsense Reasoning

Abstract

Deep learning models perform poorly on tasks that require commonsense reasoning, which often necessitates some form of worldknowledge or reasoning over information not immediately present in the input. We collect human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations (CoS-E). We use CoS-E to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation (CAGE) framework. CAGE improves the state-of-the-art by 10% on the challenging CommonsenseQA task. We further study commonsense reasoning in DNNs using both human and auto-generated explanations including transfer to out-of-domain tasks. Empirical results indicate that we can effectively leverage language models for commonsense reasoning.

Unsupervised Commonsense Question Answering with Self-Talk

“…They fine-tuned GPT2 on a question answering dataset to generate a question and an answer span for a given passage, and trained BERT to answer the generated question given the passage. Finally, Rajani et al (2019) proposed a model for Com-monSenseQA that generates explanations for its predictions. They collected human explanations and used them to fine-tune LMs to automatically generate explanations.…”

Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering

“…In the context of QA, there are multiple notions of explanation/justification, including showing an authoritative, answer-bearing sentence (Perez et al, 2019), a collection of text snippets supporting an answer (DeYoung et al, 2020), an attention map over a passage (Seo et al, 2016), a synthesized phrase connecting question and answer (Rajani et al, 2019), or the syntactic pattern used to locate the answer (Ye et al, 2020;Hancock et al, 2018). These methods are primarily designed for answers to "lookup" questions, to explain where and how an answer was found in a corpus.…”

ExpBERT: Representation Engineering with Natural Language Explanations

“…As mentioned above, semantic parsers have been used to convert language explanations into features (Srivastava et al, 2017) and noisy labels on unlabeled data (Hancock et al, 2018;Wang et al, 2019). Rather than using language to define a global collection of features, Rajani et al (2019) and Camburu et al (2018) use instance-level explanations to train models that generate their own explanations. Zaidan and Eisner (2008) ask annotators to highlight important words, then learn a generative model over parameters given these rationales.…”