Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop

Alishahi, Afra; Chrupała, Grzegorz; Linzen, Tal

doi:10.1017/s135132491900024x

Cited by 46 publications

(35 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. ] Given the success of these models, and their complexity, there is a booming interest in the computational linguistic community in understanding what aspects of language they capture, and how (Alishahi et al 2019). Recently, Pater (2019) has argued for the integration of neural network models in linguistic research (also see the responses to his article).…”

Section: Discussionmentioning

confidence: 99%

Distributional Semantics and Linguistic Theory

Boleda

2020

Annu. Rev. Linguist.

187

100

View full text Add to dashboard Cite

Distributional semantics provides multi-dimensional, graded, empirically induced word representations that successfully capture many aspects of meaning in natural languages, as shown in a large body of work in computational linguistics; yet, its impact in theoretical linguistics has so far been limited. This review provides a critical discussion of the literature on distributional semantics, with an emphasis on methods and results that are of relevance for theoretical linguistics, in three areas: semantic change, polysemy and composition, and the grammar-semantics interface (specifically, the interface of semantics with syntax and with derivational morphology). The review aims at fostering greater crossfertilization of theoretical and computational approaches to language, as a means to advance our collective knowledge of how it works.

show abstract

Section: Discussionmentioning

confidence: 99%

Distributional Semantics and Linguistic Theory

Boleda

2020

Annu. Rev. Linguist.

187

100

View full text Add to dashboard Cite

show abstract

“…How to generally and objectively evaluate explanations, without resorting to ad-hoc evaluation procedures that are domain and task specific, is still active research (Alishahi et al, 2019;Belinkov and Glass, 2019).…”

Section: Previous Workmentioning

confidence: 99%

Evaluating Recurrent Neural Network Explanations

Arras¹,

Osman²,

Müller³

et al. 2019

Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP

View full text Add to dashboard Cite

Recently, several methods have been proposed to explain the predictions of recurrent neural networks (RNNs), in particular of LSTMs. The goal of these methods is to understand the network's decisions by assigning to each input variable, e.g., a word, a relevance indicating to which extent it contributed to a particular prediction. In previous works, some of these methods were not yet compared to one another, or were evaluated only qualitatively. We close this gap by systematically and quantitatively comparing these methods in different settings, namely (1) a toy arithmetic task which we use as a sanity check, (2) a five-class sentiment prediction of movie reviews, and besides (3) we explore the usefulness of word relevances to build sentence-level representations. Lastly, using the method that performed best in our experiments, we show how specific linguistic phenomena such as the negation in sentiment analysis reflect in terms of relevance patterns, and how the relevance visualization can help to understand the misclassification of individual samples.

show abstract

“…However, their inner workings are poorly understood; indeed, for this reason, they are often referred to as black-box systems (Psichogios and Ungar, 1992;Orphanos et al, 1999;Cauer et al, 2000). This lack of understanding, coupled with the rising adoption of neural NLP systems in both industry and academia, has fomented a rapidly growing literature devoted to "cracking open the black box," as it were (Alishahi et al, 2019 et al., 2019). One popular method for studying the linguistic content of neural networks is probing, which we define in this work as training a supervised classifier (known as a probe) on top of pretrained models' frozen representations (Alain and Bengio, 2017).…”

Section: Introductionmentioning

confidence: 99%

Pareto Probing: Trading Off Accuracy for Complexity

Pimentel¹,

Saphra²,

Williams³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

The question of how to probe contextual word representations for linguistic structure in a way that is both principled and useful has seen significant attention recently in the NLP literature. In our contribution to this discussion, we argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance: the Pareto hypervolume. To measure complexity, we present a number of parametric and non-parametric metrics. Our experiments using Pareto hypervolume as an evaluation metric show that probes often do not conform to our expectations-e.g., why should the non-contextual fastText representations encode more morpho-syntactic information than the contextual BERT representations? These results suggest that common, simplistic probing tasks, such as part-of-speech labeling and dependency arc labeling, are inadequate to evaluate the linguistic structure encoded in contextual word representations. This leads us to propose full dependency parsing as a probing task. In support of our suggestion that harder probing tasks are necessary, our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.

show abstract

Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop

Cited by 46 publications

References 80 publications

Distributional Semantics and Linguistic Theory

Distributional Semantics and Linguistic Theory

Evaluating Recurrent Neural Network Explanations

Pareto Probing: Trading Off Accuracy for Complexity

Contact Info

Product

Resources

About