Intrinsic Quality Assessment of Arguments

Wachsmuth, Henning; Werner, Till

doi:10.18653/v1/2020.coling-main.592

Cited by 13 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recent works have used a structured taxonomy aiming the assessment of individual aspects based on the characteristics of the argument structure, such as the emotional appeal employed, the arrangement of the sentence, and the credibility of the message author (LAUSCHER et al, 2020;WACHSMUTH et al, 2017b;WERNER, 2020).…”

Section: Evaluating the Quality Of Argumentationmentioning

confidence: 99%

Qualidade da argumentação em tweets de política

Silva

Rassi²,

Souza

et al. 2024

Rev. Est. Ling.

View full text Add to dashboard Cite

Argumentation is something inherent to human beings and essential to written and spoken communication. Because of the popularization of Internet access, social media are one of the main means of creation and profusion of argumentative texts in various fields, such as politics. As a way to contribute to research related to the assessment of the quality of argumentation in Portuguese, we aim in this paper to propose and validate criteria and guidelines for the assessment of the quality of argumentation in Twitter posts in the domain of politics. For this purpose, a corpus was produced and annotated with tweets whose content is related to the Brazilian political scenario. The texts were collected in the first months of 2021, resulting in 1,649,674 posts. From the analysis of a sample, we defined linguistic criteria that would potentially characterize relevant aspects of the rhetorical dimension of argumentation, namely: (i) Clarity, (ii) Arrangement, (iii) Credibility, and (iv) Emotional appeal. After this phase of analysis, we proposed the annotation of a new set of 400 tweets, by four annotators. As a result, an agreement of around 70% for three out of four annotators was obtained. It is worth noting that this is the first work that proposes linguistic criteria for the evaluation of the quality of argumentation in social medias for Brazilian Portuguese. It is intended to construct a computer model that can automatically evaluate the quality of argumentation in social media messages, such as Twitter, based on the establishment of linguistic criteria, annotation rules, and annotated corpus.Keywords: argumentation; corpus; quality; rhetorical dimension; tweets; politics.Resumo: A argumentação é algo inerente ao ser humano e essencial para a comunicação escrita e falada. Por conta da popularização do acesso à Internet, as redes sociais são um dos principais meios de criação e profusão de textos argumentativos de vários domínios, como a política. Como forma de contribuir com as pesquisas relacionadas à avaliação da qualidade da argumentação em português, este trabalho tem como objetivo propor e validar critérios e diretrizes para a avaliação da qualidade da argumentação em postagens no Twitter no domínio da política. Para tanto, produziu-se um corpus anotado com tweets cujo conteúdo relaciona-se ao cenário político brasileiro. Os textos foram coletados nos primeiros meses de 2021, resultando em 1.649.674 postagens. A partir da análise de uma amostra, foram definidos critérios linguísticos que potencialmente caracterizariam aspectos relevantes da dimensão retórica da argumentação, a saber: (i) Clareza, (ii) Organização, (iii) Credibilidade e (iv) Apelo emocional. Após essa fase de análise, propôs-se a anotação de um novo conjunto de 400 tweets, por quatro anotadores. Como resultado, obteve-se uma concordância de cerca de 70% entre 3 dos 4 anotadores. Vale ressaltar que esse é o primeiro trabalho que propõe critérios linguísticos para a avaliação da qualidade da argumentação em redes sociais para o português brasileiro. A partir da definição dos critérios linguísticos, diretrizes de anotação e corpus anotado, espera-se construir um modelo computacional que possa avaliar automaticamente a qualidade da argumentação em textos de redes sociais, como o Twitter.Palavras-chave: argumentação; corpus; qualidade; dimensão retórica; tweets; política.

show abstract

Section: Evaluating the Quality Of Argumentationmentioning

confidence: 99%

Qualidade da argumentação em tweets de política

Silva

Rassi²,

Souza

et al. 2024

Rev. Est. Ling.

View full text Add to dashboard Cite

show abstract

“…Most recent research projects (Wachsmuth et al, 2017b) aim to address the challenge of redefining the notion of AQ, away from persuasiveness and towards a more "situated" definition which has to do with the needs of argumentation in a real-world scenario. This new definition has been the basis for the creation of new corpora from different domains , where feature-based (Wachsmuth and Werner, 2020) and neural models were tested for automatic prediction . Other aspects of AQ have become the subject of AM research such as the relevance and impact of arguments (Durmus et al, 2019), the verifiability (Park and Cardie, 2018), local acceptability (Yang et al, 2019) and the best "deliberative move" (Al-Khatib et al, 2018).…”

Section: Argument Quality: An Integrated Definitionmentioning

confidence: 99%

Towards Argument Mining for Social Good: A Survey

Vecchi¹,

Falk²,

Jundi³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

This survey builds an interdisciplinary picture of Argument Mining (AM), with a strong focus on its potential to address issues related to Social and Political Science. More specifically, we focus on AM challenges related to its applications to social media and in the multilingual domain, and then proceed to the widely debated notion of argument quality. We propose a novel definition of argument quality which is integrated with that of deliberative quality from the Social Science literature. Under our definition, the quality of a contribution needs to be assessed at multiple levels: the contribution itself, its preceding context, and the consequential effect on the development of the upcoming discourse. The latter has not received the deserved attention within the community. We finally define an application of AM for Social Good: (semi-)automatic moderation, a highly integrative application which (a) represents a challenging testbed for the integrated notion of quality we advocate, (b) allows the empirical quantification of argument/deliberative quality to benefit from the developments in other NLP fields (i.e. hate speech detection, fact checking, debiasing), and (c) has a clearly beneficial potential at the level of its societal thanks to its real-world application (even if extremely ambitious).

show abstract

“…They point out, that several dimensions may be perceived differently depending on the target audience. In recent follow-up work, Wachsmuth and Werner (2020) examined how well each dimension can be assessed only based on plain text only.…”

Section: Related Workmentioning

confidence: 99%

Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale

Skitalinskaya¹,

Klaff²,

Wachsmuth³

2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

Self Cite

View full text Add to dashboard Cite

Assessing the quality of arguments and of the claims the arguments are composed of has become a key task in computational argumentation. However, even if different claims share the same stance on the same topic, their assessment depends on the prior perception and weighting of the different aspects of the topic being discussed. This renders it difficult to learn topic-independent quality indicators. In this paper, we study claim quality assessment irrespective of discussed aspects by comparing different revisions of the same claim. We compile a large-scale corpus with over 377k claim revision pairs of various types from kialo.com, covering diverse topics from politics, ethics, entertainment, and others. We then propose two tasks: (a) assessing which claim of a revision pair is better, and (b) ranking all versions of a claim by quality. Our first experiments with embedding-based logistic regression and transformer-based neural networks show promising results, suggesting that learned indicators generalize well across topics. In a detailed error analysis, we give insights into what quality dimensions of claims can be assessed reliably. We provide the data and scripts needed to reproduce all results. 1

show abstract

Intrinsic Quality Assessment of Arguments

Cited by 13 publications

References 24 publications

Qualidade da argumentação em tweets de política

Qualidade da argumentação em tweets de política

Towards Argument Mining for Social Good: A Survey

Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale

Contact Info

Product

Resources

About