Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining

Abstract

The process of obtaining high quality labeled data for natural language understanding tasks is often slow, error-prone, complicated and expensive. With the vast usage of neural networks, this issue becomes more notorious since these networks require a large amount of labeled data to produce satisfactory results. We propose a methodology to blend high quality but scarce labeled data with noisy but abundant weak labeled data during the training of neural networks. Experiments in the context of topic-dependent evidence detection with two forms of weak labeled data show the advantages of the blending scheme. In addition, we provide a manually annotated data set for the task of topicdependent evidence detection.

Classification and Clustering of Arguments with Contextualized Word Embeddings

“…Both BERT-base and BERT-large outperform the baseline IBM set by Shnarch et al (2018) already by more than 6pp in accuracy 5 . The topic integrating models IBM ELM o and IBM BERT do not improve much over their BiLSTM counterparts, which do not use any topic information.…”

“…A macro F 1 -score is computed for the 3-label classes and scores are averaged over all topics and over ten random seeds. For the IBM Corpus, we use the setup by Shnarch et al (2018): Training on 83 topics (4,066 sentences) and testing on 35 topics (1,719 sentences). We train for five different random seeds and report the average accuracy over all runs.…”

“…UKP Corpus. The UKP Sentential Argument Shnarch et al (2018) extracted sentences from Wikidata that were in turn annotated by crowd-workers (10 for each topic-sentence pair) with one of the two labels: evidence or no evidence in regard to the topic.…”

“…We thus analyze ELMo and BERT in a cross-topic scenario for the tasks of argument classification and clustering on four different datasets. For argument classification, we use the UKP Sentential Argument Mining Corpus by Stab et al (2018b) and the IBM Debater R : Evidence Sentences corpus by Shnarch et al (2018). For argument clustering, we introduce a novel corpus on aspect-based argument clustering and evaluate the proposed methods on this corpus as well as on the Argument Facet Similarity Corpus (Misra et al, 2016).…”

The evolution of argumentation mining: From models to social media and emerging tools

“…The process of manual annotation is labor intensive and time consuming, whereas possible use of unsupervised machine learning algorithms could solve the problem of lack of trained annotators. The need for novel algorithms and techniques is emphasized also in previous review papers [12,37] and although there is an evident trend towards unsupervised [132,133] or semi-supervised ML algorithms [7,20,129,134] with notable performance, they could be further improved, as there has not been extensive work neither on the use of suitable features nor on the design of argument schemes. A recent literature review from Silva et al [135] presents the trends in semi-supervised learning for tweet sentiment analysis and it can be used as a point of reference also in the field of AM.…”