oLMpics-On What Language Model Pre-training Captures

Abstract

Recent success of pre-trained language models (LMs) has spurred widespread interest in the language capabilities that they possess. However, efforts to understand whether LM representations are useful for symbolic reasoning tasks have been limited and scattered. In this work, we propose eight reasoning tasks, which conceptually require operations such as comparison, conjunction, and composition. A fundamental challenge is to understand whether the performance of a LM on a task should be attributed to the pre-trained representations or to the process of fine-tuning on the task data. To address this, we propose an evaluation protocol that includes both zero-shot evaluation (no fine-tuning), as well as comparing the learning curve of a fine-tuned LM to the learning curve of multiple controls, which paints a rich picture of the LM capabilities. Our main findings are that: (a) different LMs exhibit qualitatively different reasoning abilities, e.g., RoBERTa succeeds in reasoning tasks where BERT fails completely; (b) LMs do not reason in an abstract manner and are context-dependent, e.g., while RoBERTa can compare ages, it can do so only when the ages are in the typical range of human ages; (c) On half of our reasoning tasks all models fail completely. Our findings and infrastructure can help future work on designing new datasets, models, and objective functions for pre-training.

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

“…These works cover a wide variety of topics, including grammatical generalization(Goldberg, 2019;Warstadt et al, 2019), syntax(Tenney et al, 2019b;Lin et al, 2019;Reif et al, 2019;Hewitt and Manning, 2019;Liu et al, 2019a), world knowledge(Petroni et al, 2019;Jiang et al, 2020), reasoning(Talmor et al, 2019), and common sense(Forbes et al, 2019;Zhou et al, 2019;Weir et al, 2020). 169 Downloaded from http://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00359/1894330/tacl_a_00359.pdf by guest on 10 May 2021…”

Probing Contextual Language Models for Common Ground with Visual Representations

“…Understanding what information NLP models encode has attracted great interest in recent years (Rogers et al, 2020). From factual (Petroni et al, 2019;Jawahar et al, 2019;Roberts et al, 2020) to linguistic (Conneau et al, 2018;Liu et al, 2019a;Talmor et al, 2019) and commonsense (Forbes et al, 2019) knowledge, a wide set of properties have been previously analysed. We refer to Belinkov and Glass (2019) and Rogers et al (2020) for a more comprehensive literature review.…”

What BERTs and GPTs know about your brand? Probing contextual language models for affect associations

“…If a model predicts the correct attribute in the top-5 position, then we infer that the model representations have captured the corresponding affect association. Additionally, to understand the behavior after fine-tuning, we introduce MLP with a 1-hidden layer to the MLM setup to train the LMs as discussed in (Talmor et al, 2019); we call this setup MLP-MLM.…”

“…Dataset and the BCD datasets (RQ1). Further, to understand the impact of fine-tuning, we employ techniques proposed by (Talmor et al, 2019) to measure the language mismatch. In this exercise, we fine-tune the LM with examples from Ads.…”

“…We believe that diversity in architectures and training data can influence the affective associations stored in representations. We propose to evaluate word representations based on following dimensions: 1) contextual similarity (Ethayarajh, 2019), 2) statistical implicit association tests (Kurita et al, 2019;, 3) controlled probing tasks (Talmor et al, 2019) and 4) brand archetypes (Bechter et al, 2016). We observe that LMs do encode affective associations between brands and image attributes (RQ1).…”