Untitled

Talmor, Alon; Herzig, Jonathan; Lourie, Nicholas; Berant, Jonathan

doi:10.18653/v1/n19-1421

Cited by 321 publications

(267 citation statements)

References 21 publications

Supporting

Mentioning

264

Contrasting

Order By: Relevance

“…Across NLP, a lot of work has been published around different kinds of reasonings. To name a few, common sense (Talmor et al, 2019), temporal (Zhou et al, 2019), numerical (Naik et al, 2019;Wallace et al, 2019b) and multi-hop (Khashabi et al, 2018) reasoning have all garnered immense research interest.…”

Section: Resultsmentioning

confidence: 99%

INFOTABS: Inference on Tables as Semi-structured Data

Gupta¹,

Mehta²,

Nokhiz³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

In this paper, we observe that semi-structured tabulated text is ubiquitous; understanding them requires not only comprehending the meaning of text fragments, but also implicit relationships between them. We argue that such data can prove as a testing ground for understanding how we reason about information. To study this, we introduce a new dataset called INFOTABS, comprising of human-written textual hypotheses based on premises that are tables extracted from Wikipedia info-boxes. Our analysis shows that the semi-structured, multi-domain and heterogeneous nature of the premises admits complex, multi-faceted reasoning. Experiments reveal that, while human annotators agree on the relationships between a table-hypothesis pair, several standard modeling strategies are unsuccessful at the task, suggesting that reasoning about tables can pose a difficult modeling challenge.

show abstract

Section: Resultsmentioning

confidence: 99%

INFOTABS: Inference on Tables as Semi-structured Data

Gupta¹,

Mehta²,

Nokhiz³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…While recent advances in large-scale neural language models Radford et al, 2019;Raffel et al, 2019) have led to strong performance on several commonsense reasoning benchmarks (Talmor et al, 2019;Lv et al, 2020;Sakaguchi et al, 2020), their accuracy by and large depends on the availability of large-scale human-authored training data. However, crowdsourcing examples at scale for each new task and domain can be prohibitively expensive.…”

Section: Introductionmentioning

confidence: 99%

Generative Data Augmentation for Commonsense Reasoning

Yang

Malaviya²,

Fernandez

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Recent advances in commonsense reasoning depend on large-scale human-annotated training sets to achieve peak performance. However, manual curation of training sets is expensive and has been shown to introduce annotation artifacts that neural models can readily exploit and overfit to. We propose a novel generative data augmentation technique, G-DAUG c , that aims to achieve more accurate and robust learning in a low-resource setting. Our approach generates synthetic examples using pretrained language models, and selects the most informative and diverse set of examples for data augmentation. On experiments with multiple commonsense reasoning benchmarks, G-DAUG c consistently outperforms existing data augmentation methods based on back-translation, establishing a new state-of-the-art on WINOGRANDE, CODAH, and COMMONSENSEQA, and also enhances out-of-distribution generalization, proving to be more robust against adversaries or perturbations. Our analysis demonstrates that G-DAUG c produces a diverse set of fluent training examples, and that its selection and training approaches are important for performance.

show abstract

“…Using this input format, we apply a new bidirectional word-level scoring function that leverages the MLM head (Devlin et al, 2019) tuned during the pre-training phase (see Figure 1 for an overview of the proposed approach). This method produces strong zero-shot 1 baselines on the COPA (Gordon et al, 2012), Swag (Zellers et al, 2018), HellaSwag (Zellers et al, 2019) and CommonsenseQA (Talmor et al, 2019) datasets. Then, we fine-tune this new scoring function with a margin-based loss as proposed in .…”

Section: Introductionmentioning

confidence: 99%

Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning

Tamborrino¹,

Pellicanò²,

Pannier³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks (Devlin et al., 2019). Most of the existing approaches rely on a randomly initialized classifier on top of such networks. We argue that this fine-tuning procedure is sub-optimal as the pre-trained model has no prior on the specific classifier labels, while it might have already learned an intrinsic textual representation of the task. In this paper, we introduce a new scoring method that casts a plausibility ranking task in a full-text format and leverages the masked language modeling head tuned during the pre-training phase. We study commonsense reasoning tasks where the model must rank a set of hypotheses given a premise, focusing on the COPA (Gordon et al., 2012), Swag (Zellers et al., 2018), HellaSwag (Zellers et al., 2019) and CommonsenseQA (Talmor et al., 2019) datasets. By exploiting our scoring method without fine-tuning, we are able to produce strong baselines (e.g. 80% test accuracy on COPA) that are comparable to supervised approaches. Moreover, when fine-tuning directly on the proposed scoring function, we show that our method provides a much more stable training phase across random restarts (e.g ×10 standard deviation reduction on COPA test accuracy) and requires less annotated data than the standard classifier approach to reach equivalent performances.

show abstract

Untitled

Cited by 321 publications

References 21 publications

INFOTABS: Inference on Tables as Semi-structured Data

INFOTABS: Inference on Tables as Semi-structured Data

Generative Data Augmentation for Commonsense Reasoning

Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning

Contact Info

Product

Resources

About