Relative Importance in Sentence Processing

Hollenstein, Nora; Beinborn, Lisa

doi:10.18653/v1/2021.acl-short.19

Cited by 15 publications

(8 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This new eye tracking dataset also allows us to analyze and interpret language models or task-specific NLP models. For example, we can investigate machine-learning based explainability mechanisms such as attention and saliency in Danish language models, as suggested by Hollenstein and Beinborn (2021) or Sood et al (2020) for English.…”

Section: Discussionmentioning

confidence: 99%

The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts

Hollenstein¹,

Barrett²,

Björnsdóttir³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Eye movement recordings from reading are one of the richest signals of human language processing. Corpora of eye movements during reading of contextualized running text is a way of making such records available for natural language processing purposes. Such corpora already exist in some languages. We present CopCo, the Copenhagen Corpus of eye tracking recordings from natural reading of Danish texts. It is the first eye tracking corpus of its kind for the Danish language. CopCo includes 1,832 sentences with 34,897 tokens of Danish text extracted from a collection of speech manuscripts. This first release of the corpus contains eye tracking data from 22 participants. It will be extended continuously with more participants and texts from other genres. We assess the data quality of the recorded eye movements and find that the extracted features are in line with related research. The dataset available here: https://osf.io/ud8s5/.

show abstract

Section: Discussionmentioning

confidence: 99%

The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts

Hollenstein¹,

Barrett²,

Björnsdóttir³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, we compute averages over last-layer attention head vectors ('Attention-last') as proposed in Hollenstein and Beinborn [2021] as well as 'Rollout' and attention flow ('A-flow') Abnar and Zuidema [2020] which capture the layerwise structure of deep Transformer models in comparison to raw attention head analysis.…”

Section: Benchmark Methodsmentioning

confidence: 99%

XAI for Transformers: Better Explanations through Conservative Propagation

Ameen¹,

Schnake²,

Eberle³

et al. 2022

Preprint

View full text Add to dashboard Cite

Transformers have become an important workhorse of machine learning, with numerous applications. This necessitates the development of reliable methods for increasing their transparency. Multiple interpretability methods, often based on gradient information, have been proposed. We show that the gradient in a Transformer reflects the function only locally, and thus fails to reliably identify the contribution of input features to the prediction. We identify Attention Heads and LayerNorm as main reasons for such unreliable explanations and propose a more stable way for propagation through these layers. Our proposal, which can be seen as a proper extension of the well-established LRP method to Transformers, is shown both theoretically and empirically to overcome the deficiency of a simple gradient-based approach, and achieves state-of-the-art explanation performance on a broad range of Transformer models and datasets.

show abstract

“…For the present benchmark, this is possible by leveraging a naturalistic dataset of reading English sentences, the Zurich Cognitive Language Processing Corpus (Hollenstein et al, 2018 , 2020 ). The ZuCo dataset is publicly available and has recently been used in a variety of applications including leveraging EEG and eye-tracking data to improve NLP tasks (Barrett et al, 2018 ; Mathias et al, 2020 ; McGuire and Tomuro, 2021 ), evaluating the cognitive plausibility of computational language models (Hollenstein et al, 2019b ; Hollenstein and Beinborn, 2021 ), investigating the neural dynamics of reading (Pfeiffer et al, 2020 ), developing models of human reading (Bautista and Naval, 2020 ; Bestgen, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

Hollenstein

Tröndle²,

Płomecka³

et al. 2023

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com.

show abstract

Relative Importance in Sentence Processing

Cited by 15 publications

References 43 publications

The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts

The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts

XAI for Transformers: Better Explanations through Conservative Propagation

The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

Contact Info

Product

Resources

About