Spurious Correlations in Cross-Topic Argument Mining

Jakobsen, Terne Sasha Thorn; Barrett, Maria; Søgaard, Anders

doi:10.18653/v1/2021.starsem-1.25

Cited by 3 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As evidenced by several studies that involved manual annotation of texts (Stab and Gurevych, 2014;Kirschner et al, 2015;Habernal and Gurevych, 2017), there is very often disagreement between annotators on the arguments, components of arguments or argument relations conveyed by a text, which in most cases is due to the ambiguity of human language. As shown in Thorn Jakobsen et al (2022), it may also be due to the different backgrounds and demographic characteristics of the annotators. Manual annotation may therefore introduce social bias to the data used to train data-driven argument mining methods and, as a result, also to the methods themselves.…”

Section: Extraction and Annotationmentioning

confidence: 99%

Sketching the vision of the Web of Debates

Bikakis

Flouris

Patkos

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

The exchange of comments, opinions, and arguments in blogs, forums, social media, wikis, and review websites has transformed the Web into a modern agora, a virtual place where all types of debates take place. This wealth of information remains mostly unexploited: due to its textual form, such information is difficult to automatically process and analyse in order to validate, evaluate, compare, combine with other types of information and make it actionable. Recent research in Machine Learning, Natural Language Processing, and Computational Argumentation has provided some solutions, which still cannot fully capture important aspects of online debates, such as various forms of unsound reasoning, arguments that do not follow a standard structure, information that is not explicitly expressed, and non-logical argumentation methods. Tackling these challenges would give immense added-value, as it would allow searching for, navigating through and analyzing online opinions and arguments, obtaining a better picture of the various debates for a well-intentioned user. Ultimately, it may lead to increased participation of Web users in democratic, dialogical interchange of arguments, more informed decisions by professionals and decision-makers, as well as to an easier identification of biased, misleading, or deceptive arguments. This paper presents the vision of the Web of Debates, a more human-centered version of the Web, which aims to unlock the potential of the abundance of argumentative information that currently exists online, offering its users a new generation of argument-based web services and tools that are tailored to their real needs.

show abstract

Section: Extraction and Annotationmentioning

confidence: 99%

Sketching the vision of the Web of Debates

Bikakis

Flouris

Patkos

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Cross-domain classification has been investigated in NLP tasks such as sentiment analysis (Al-Moslmi et al, 2017;Qu et al, 2019;Du et al, 2020), fake news detection (Fung et al, 2021;Silva et al, 2021;Yuan et al, 2021), and argument mining (Al-Khatib et al, 2016;Daxenberger et al, 2017;Thorn Jakobsen et al, 2021). These tasks are similar to value classification in that they aim to classify high-level constructs (such as sentiments and arguments).…”

Section: Cross-domain Nlp Classificationmentioning

confidence: 99%

Cross-Domain Classification of Moral Values

Liscio¹,

Dondera²,

Geadau³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

Moral values influence how we interpret and act upon the information we receive. Identifying human moral values is essential for artificially intelligent agents to co-exist with humans. Recent progress in natural language processing allows the identification of moral values in textual discourse. However, domain-specific moral rhetoric poses challenges for transferring knowledge from one domain to another.We provide the first extensive investigation on the effects of cross-domain classification of moral values from text. We compare a state-ofthe-art deep learning model (BERT) in seven domains and four cross-domain settings. We show that a value classifier can generalize and transfer knowledge to novel domains, but it can introduce catastrophic forgetting. We also highlight the typical classification errors in crossdomain value classification and compare the model predictions to the annotators agreement. Our results provide insights to computer and social scientists that seek to identify moral rhetoric specific to a domain of discourse.

show abstract

“…However, applying AM methods for feedback analysis poses three main challenges. First, AM methods generalize poorly across domains [5,6]. Thus, they require large amounts of domain-specific training data, which is often not available.…”

Section: Introductionmentioning

confidence: 99%

HyEnA: A Hybrid Method for Extracting Arguments from Opinions

Meer

Liscio

Jonker

et al. 2022

HHAI2022: Augmenting Human Intellect

View full text Add to dashboard Cite

The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.

show abstract

Spurious Correlations in Cross-Topic Argument Mining

Cited by 3 publications

References 26 publications

Sketching the vision of the Web of Debates

Sketching the vision of the Web of Debates

Cross-Domain Classification of Moral Values

HyEnA: A Hybrid Method for Extracting Arguments from Opinions

Contact Info

Product

Resources

About