Intrinsic Probing through Dimension Selection

Hennigen, Lucas Torroba; Williams, Adina; Cotterell, Ryan

doi:10.18653/v1/2020.emnlp-main.15

Cited by 32 publications

(27 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the Gaussian distribution is the maximum entropy distribution given a mean and covariance matrix, it makes the fewest assumptions and is therefore a reasonable default. Hennigen et al (2020) found that embeddings sometimes do not follow a Gaussian distribution, but it is unclear what alternative distribution would be a better fit, so we will assume a Gaussian distribution in this work.…”

Section: Connection To Mahalanobis Distancementioning

confidence: 99%

How is BERT surprised? Layerwise detection of linguistic anomalies

Li¹,

Zhu²,

Thomas³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Transformer language models have shown remarkable ability in detecting when a word is anomalous in context, but likelihood scores offer no information about the cause of the anomaly. In this work, we use Gaussian models for density estimation at intermediate layers of three language models (BERT, RoBERTa, and XLNet), and evaluate our method on BLiMP, a grammaticality judgement benchmark. In lower layers, surprisal is highly correlated to low token frequency, but this correlation diminishes in upper layers. Next, we gather datasets of morphosyntactic, semantic, and commonsense anomalies from psycholinguistic studies; we find that the best performing model RoBERTa exhibits surprisal in earlier layers when the anomaly is morphosyntactic than when it is semantic, while commonsense anomalies do not exhibit surprisal at any intermediate layer. These results suggest that language models employ separate mechanisms to detect different types of linguistic anomalies.

show abstract

Section: Connection To Mahalanobis Distancementioning

confidence: 99%

How is BERT surprised? Layerwise detection of linguistic anomalies

Li¹,

Zhu²,

Thomas³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Alain and Bengio, 2016;Hewitt and Manning, 2019;Hall Maudslay et al, 2020) or a subset of neurons at a time (e.g. Torroba Hennigen et al, 2020;Mu and Andreas, 2020;Durrani et al, 2020). However, restricting our analysis this way seems arbitrary.…”

Section: Ease Of Extraction and Previous Workmentioning

confidence: 99%

A Bayesian Framework for Information-Theoretic Probing

Pimentel¹,

Cotterell²

2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Pimentel et al. (2020b) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. The mutual information, however, assumes the true probability distribution of a pair of random variables is known, leading to unintuitive results in settings where it is not. This paper proposes a new framework to measure what we term Bayesian mutual information, which analyses information from the perspective of Bayesian agents-allowing for more intuitive findings in scenarios with finite data. For instance, under Bayesian MI we have that data can add information, processing can help, and information can hurt, which makes it more intuitive for machine learning applications. Finally, we apply our framework to probing where we believe Bayesian mutual information naturally operationalises ease of extraction by explicitly limiting the available background knowledge to solve a task.

show abstract

“…The first part of the tutorial covers methods that align neurons to human interpretable concepts or study the most salient neurons in the network. We cluster these methods into four groups i) Visualization Methods (Karpathy et al, 2015;Li et al, 2016a), ii) Corpus Selection (Kádár et al, 2017;Poerner et al, 2018;Na et al, 2019;Mu and Andreas, 2020b), iii) Neuron Probing (Dalvi et al, 2019a;Lakretz et al, 2019;Valipour et al, 2019;Durrani et al, 2020) and iv) Unsupervised Methods (Bau et al, 2019;Torroba Hennigen et al, 2020;Michael et al, 2020). We will discuss evaluation methods that are used to measure the effectiveness of an interpretation method, such as accuracy, control tasks (Hewitt and Liang, 2019) and ablation studies (Li et al, 2016b;Lillian et al, 2018;Dalvi et al, 2019a;Lakretz et al, 2019).…”

Section: Descriptionmentioning

confidence: 99%

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials

2021

View full text Add to dashboard Cite

The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query for a particular task. Although the most common formulation of text ranking is search, instances of the task can also be found in many text processing applications. This tutorial provides an overview of text ranking with neural network architectures known as transformers, of which BERT (Bidirectional Encoder Representations from Transformers) (Devlin et al., 2019) is the best-known example. These models produce high quality results across many domains, tasks, and settings.This tutorial, which is based on the preprint (Lin et al., 2020a) of a forthcoming book to be published by Morgan and & Claypool under the Synthesis Lectures on Human Language Technologies series, provides an overview of existing work as a single point of entry for practitioners who wish to deploy transformers for text ranking in real-world applications and researchers who wish to pursue work in this area. We cover a wide range of techniques, grouped into two categories: transformer models that perform reranking in multi-stage ranking architectures and learned dense representations that perform ranking directly.

show abstract

Intrinsic Probing through Dimension Selection

Cited by 32 publications

References 46 publications

How is BERT surprised? Layerwise detection of linguistic anomalies

How is BERT surprised? Layerwise detection of linguistic anomalies

A Bayesian Framework for Information-Theoretic Probing

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials

Contact Info

Product

Resources

About