Information-Theoretic Probing with Minimum Description Length

Voita, Elena; Titov, Ivan

doi:10.18653/v1/2020.emnlp-main.14

Cited by 166 publications

(265 citation statements)

References 31 publications

Supporting

Mentioning

257

Contrasting

Order By: Relevance

“…Then, they could be not as readily "available" after finetuning. We explored this hypothesis using Minimum Description Length probes (Voita and Titov, 2020), with the results presented in Appendix B. We found minimal differences across most tasks, where the only significant result was that finetuning on dependency parsing made the corresponding edge probing task easier to learn as a function of the number of examples.…”

Section: Resultsmentioning

confidence: 99%

“…Tenney et al (2019b); ; Peters et al (2018b) introduced task suites that probe for high-level linguistic phenomena such as partof-speech, entity types, and coreference, while Tenney et al (2019a) showed that these phenomena are represented in a hierarchical order within the layers of BERT. Hewitt and Manning (2019) used a geometrically-motivated probe to explore syntactic structures, and Voita and Titov (2020) and Pimentel et al (2020) designed informationtheoretic techniques that can measure the model and data complexity. 1 While probing models depend on labelled data, parallel work has studied the same encoders using unsupervised techniques.…”

Section: Related Workmentioning

confidence: 99%

“…We approach these questions with three complementary analysis techniques. Supervised probing classifiers (Tenney et al, 2019b;Hewitt and Manning, 2019;Voita and Titov, 2020) provide a means of explicitly testing for the presence of pre-specified linguistic phenomena, while Representational Similarity Analysis (RSA; Kriegeskorte et al, 2008) gives a task-agnostic measurement of the change in model activations. Finally, we corroborate our results with two types of layerbased ablations-truncation and partial freezingand measure their effect on end-task performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

What Happens To BERT Embeddings During Fine-tuning?

Merchant¹,

Rahimtoroghi²,

Pavlick³

et al. 2020

Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

While much recent work has examined how linguistic information is encoded in pretrained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks.Using a suite of analysis techniques-supervised probing, unsupervised similarity analysis, and layer-based ablations-we investigate how fine-tuning affects the representations of the BERT model. We find that while fine-tuning necessarily makes some significant changes, there is no catastrophic forgetting of linguistic phenomena. We instead find that fine-tuning is a conservative process that primarily affects the top layers of BERT, albeit with noteworthy variation across tasks. In particular, dependency parsing reconfigures most of the model, whereas SQuAD and MNLI involve much shallower processing. Finally, we also find that fine-tuning has a weaker effect on representations of out-of-domain sentences, suggesting room for improvement in model generalization.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

What Happens To BERT Embeddings During Fine-tuning?

Merchant¹,

Rahimtoroghi²,

Pavlick³

et al. 2020

Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

show abstract

“…Given that the Transformer decoder has far more parameters than either probe, it is not surprising that less direct representations of some linguistic features would not impact translation performance. Differences in how directly sparse & dense models encode information is an interesting question perhaps well-suited to recent work on minimum description lengths (Voita and Titov, 2020), but we leave it to future study. Meanwhile, the MLP could not rescue sparse model performance on PS-Fxn, PS-Role, and Coref; results were nearly identical as with the linear probe.…”

Section: Are Results Probe-sensitive?mentioning

confidence: 99%

Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation

Rajiv

Zhao

2020

Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

Recent work on the lottery ticket hypothesis has produced highly sparse Transformers for NMT while maintaining BLEU. However, it is unclear how such pruning techniques affect a model's learned representations. By probing Transformers with more and more lowmagnitude weights pruned away, we find that complex semantic information is first to be degraded. Analysis of internal activations reveals that higher layers diverge most over the course of pruning, gradually becoming less complex than their dense counterparts. Meanwhile, early layers of sparse models begin to perform more encoding. Attention mechanisms remain remarkably consistent as sparsity increases.

show abstract

“…Some other work also investigates models' ability to induce syntactic information by measuring accuracy of a probe (Zhang and Bowman, 2018;Hewitt and Manning, 2019;Hewitt and Liang, 2019). However, there is significant uncertainty about how to calibrate such probing results (Voita and Titov, 2020); our model's representations are more directly interpretable and don't require posthoc probing.…”

Section: Analysis: Entity Typing Performancementioning

confidence: 99%

Interpretable Entity Representations through Large-Scale Typing

Onoe

Durrett

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

In standard methodology for natural language processing, entities in text are typically embedded in dense vector spaces with pre-trained models. The embeddings produced this way are effective when fed into downstream models, but they require end-task fine-tuning and are fundamentally difficult to interpret. In this paper, we present an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. Our representations are vectors whose values correspond to posterior probabilities over finegrained entity types, indicating the confidence of a typing model's decision that the entity belongs to the corresponding type. We obtain these representations using a fine-grained entity typing model, trained either on supervised ultra-fine entity typing data (Choi et al., 2018) or distantly-supervised examples from Wikipedia. On entity probing tasks involving recognizing entity identity, our embeddings used in parameter-free downstream models achieve competitive performance with ELMoand BERT-based embeddings in trained models. We also show that it is possible to reduce the size of our type set in a learning-based way for particular domains. Finally, we show that these embeddings can be post-hoc modified through a small number of rules to incorporate domain knowledge and improve performance.

show abstract

Information-Theoretic Probing with Minimum Description Length

Cited by 166 publications

References 31 publications

What Happens To BERT Embeddings During Fine-tuning?

What Happens To BERT Embeddings During Fine-tuning?

Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation

Interpretable Entity Representations through Large-Scale Typing

Contact Info

Product

Resources

About