Recognizing Insufficiently Supported Arguments in Argumentative Essays

Abstract

In this paper, we propose a new task for assessing the quality of natural language arguments. The premises of a well-reasoned argument should provide enough evidence for accepting or rejecting its claim. Although this criterion, known as sufficiency, is widely adopted in argumentation theory, there are no empirical studies on its applicability to real arguments. In this work, we show that human annotators substantially agree on the sufficiency criterion and introduce a novel annotated corpus. Furthermore, we experiment with feature-rich SVMs and convolutional neural networks and achieve 84% accuracy for automatically identifying insufficiently supported arguments. The final corpus as well as the annotation guideline are freely available for encouraging future research on argument quality. 1

Computational Argumentation Quality Assessment in Natural Language

“…Also, we omit approaches that classify argumentation schemes (Feng and Hirst, 2011), evidence types (Rinott et al, 2015), ethosrelated statements (Duthie et al, 2016), and myside bias ; their output may help assess quality assessment, but they do not actually assess it. The same holds for argument mining, Level of support Braunstain et al (2016) Evidence Rahimi et al (2014) Sufficiency Stab and Gurevych (2017) Thesis clarity Persing and Ng (2013) Prompt adherence Persing and Ng (2014) Global coherence Feng et al (2014) Evaluability Park et al (2015) Acceptability Cabrio and Villata (2012) Organization Persing et al (2010), Rahimi et al (2015) Argument strength Persing et al (2015) Persuasiveness Tan et al (2016), Wei et al (2016) Winning side Zhang et al (2016) Convincingness Habernal et al (2016) Prominence Boltužic and Šnajder (2015) Relevance Wachsmuth et al (2017) Figure 1: The proposed taxonomy of argumentation quality as well as the mapping of existing assessment approaches to the covered quality dimensions. Arrows show main dependencies between the dimensions.…”

Incongruent Headlines: Yet Another Way to Mislead Your Readers

“…It may also be interesting to explore existing work on argument analysis: for example, Stab and Gurevych (2017) explore methods for the identification of arguments supported by insufficient evidence. This could be viewed as very close to the task of the detection of incongruent headlines, where the headline represents an argument which is not supported by claims in the text.…”

Before Name-Calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation

“…The topic of fallacies, which might be considered as sub-topic of argumentation quality, has recently been investigated also in the NLP field. Existing works are, however, limited to the monological view (Wachsmuth et al, 2017;Habernal and Gurevych, 2016b,a;Stab and Gurevych, 2017) or they focus primarily on learning fallacy recognition by humans (Habernal et al, , 2018a. Another related NLP sub-field includes abusive language and personal attacks in general.…”