Is It Worth the Attention? A Comparative Evaluation of Attention Layers for Argument Unit Segmentation

Spliethöver, Maximilian; Klaff, Jonas; Heuer, Hendrik

doi:10.18653/v1/w19-4509

Cited by 11 publications

(12 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An empirical evaluation is thus beyond the scope of this article. There are, however, a number of experimental studies focused on particular NLP tasks, including machine translation [37], [42], [48], [132], argumentation mining [125], text summarization [58], and sentiment analysis [7]. It is worthwhile remarking that, on several occasions, attention-based approaches enabled a dramatic development of entire research lines.…”

Section: Introductionmentioning

confidence: 99%

Attention in Natural Language Processing

Galassi

Lippi

Torroni

2021

IEEE Trans. Neural Netw. Learning Syst.

449

130

View full text Add to dashboard Cite

Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.

show abstract

Section: Introductionmentioning

confidence: 99%

Attention in Natural Language Processing

Galassi

Lippi

Torroni

2021

IEEE Trans. Neural Netw. Learning Syst.

449

130

View full text Add to dashboard Cite

show abstract

“…The authors noted an decreased number of invalid BI sequences with the addition of the second (upper) BiL-STM. In later work, the authors in [33] further investigated this architecture with minor changes: they used solely one BiLSTM with word embeddings as input features and tested the efficacy of the second (upper) BiLSTM. Moreover, they investigated the effects of adding various attention layers.…”

Section: Related Workmentioning

confidence: 99%

Enhancing evidence-based medicine with natural language argumentative analysis of clinical trials

Mayer

Marro

Cabrio

et al. 2021

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

In the latest years, the healthcare domain has seen an increasing interest in the definition of intelligent systems to support clinicians in their everyday tasks and activities. Among others, also the field of Evidence-Based Medicine is impacted by this twist, with the aim to combine the reasoning frameworks proposed thus far in the field with mining algorithms to extract structured information from clinical trials, clinical guidelines, and Electronic Health Records. In this paper, we go beyond the state of the art by proposing a new end-to-end pipeline to address argumentative outcome analysis on clinical trials. More precisely, our pipeline is composed of (i) an Argument Mining module to extract and classify argumentative components (i.e., evidence and claims of the trial) and their relations (i.e., support, attack), and (ii) an outcome analysis module to identify and classify the effects (i.e., improved, increased, decreased, no difference, no occurrence) of an intervention on the outcome of the trial, based on PICO elements. We annotated a dataset composed of more than 500 abstracts of Randomized Controlled Trials (RCT) from the MEDLINE database, leading to a labeled dataset with 4198 argument components, 2601 argument relations, and 3351 outcomes on five different diseases (i.e., neoplasm, glaucoma, hepatitis, diabetes, hypertension). We experiment with deep bidirectional transformers in combination with different neural architectures (i.e., LSTM, GRU and CRF) and obtain a macro F1-score of .87 for component detection and .68 for relation prediction, outperforming current state-of-the-art end-to-end Argument Mining systems, and a macro F1-score of .80 for outcome classification.

show abstract

“…PE 2.0 annotates three kinds of argumentation components, namely major claim (MC), claim (C) and premise (P). Many previous researches (Persing and Ng, 2016;Chernodub et al, 2019;Petasis, 2019;Reimers et al, 2019;Spliethover et al, 2019)…”

Section: Datasetmentioning

confidence: 99%

“…Chernodub et al (2019) tried to build application interface, which is called TARGER and is a BiLSTM-CNN-CRF sequence tagging model, for convenient argumentation mining on essays. Besides, latest research (Petasis, 2019;Spliethover et al, 2019) also aims to distinguish argumentation components from non-argumentation components with text segmentation based on sequence tagging models. Other work Peldszus et al, 2016;Skeppstedt et al, 2018) focuses on arg-microtext corpus , which contains 112 independent short texts, where each can be considered as one paragraph and contains about 5 argumentation components on average.…”

Section: Argumentation Extraction or Argumentation Taggingmentioning

confidence: 99%

Argumentation Mining on Essays at Multi Scales

Wang¹,

Huang²,

Dou³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Argumentation mining on essays is a new challenging task in natural language processing, which aims to identify the types and locations of argumentation components. Recent research mainly models the task as a sequence tagging problem and deal with all the argumentation components at word level. However, this task is not scale-independent. Some types of argumentation components which serve as core opinions on essays or paragraphs, are at essay level or paragraph level. Sequence tagging method conducts reasoning by local context words, and fails to effectively mine these components. To this end, we propose a multi-scale argumentation mining model, where we respectively mine different types of argumentation components at corresponding levels. Besides, an effective coarse-to-fine argumentation fusion mechanism is proposed to further improve the performance. We conduct a serial of experiments on the Persuasive Essay dataset (PE 2.0). Experimental results indicate that our model outperforms existing models on mining all types of argumentation components.

show abstract

Is It Worth the Attention? A Comparative Evaluation of Attention Layers for Argument Unit Segmentation

Cited by 11 publications

References 17 publications

Attention in Natural Language Processing

Attention in Natural Language Processing

Enhancing evidence-based medicine with natural language argumentative analysis of clinical trials

Argumentation Mining on Essays at Multi Scales

Contact Info

Product

Resources

About