Explainable Unsupervised Argument Similarity Rating with Abstract Meaning Representation and Conclusion Generation

Abstract

When assessing the similarity of arguments, researchers typically use approaches that do not provide interpretable evidence or justifications for their ratings. Hence, the features that determine argument similarity remain elusive. We address this issue by introducing novel argument similarity metrics that aim at high performance and explainability. We show that Abstract Meaning Representation (AMR) graphs can be useful for representing arguments, and that novel AMR graph metrics can offer explanations for argument similarity ratings. We start from the hypothesis that similar premises often lead to similar conclusionsand extend an approach for AMR-based argument similarity rating by estimating, in addition, the similarity of conclusions that we automatically infer from the arguments used as premises. We show that AMR similarity metrics make argument similarity judgements more interpretable and may even support argument quality judgements. Our approach provides significant performance improvements over strong baselines in a fully unsupervised setting. Finally, we make first steps to address the problem of reference-less evaluation of argumentative conclusion generations.

Proceedings of the 15th International Conference on Natural Language Generation

“…We follow previous work by Opitz et al (2021) and rely on T5 (Raffel et al, 2020) (large version) as transformer language model, as implemented in the huggingface library (Wolf et al, 2020).…”

“…by rephrasing the premise, but only 4-6% are informative. Opitz et al (2021) also show that state-of-the-art fine-tuned transformer language models processing plain premises tend to generate conclusions lacking in novelty or validity, and proposed ways to assess their novelty and validity using AMR-based similarity metrics. Finally, Gurcke et al (2021) explored whether the sufficiency of conclusions can be assessed with BART, and find problems with insufficient reference conclusions -with ensuing challenges in generating and evaluating valid and novel conclusions.…”

“…Such omissions are frequent and often due to rhetorical reasons (Rajendran et al, 2016;Becker et al, 2021). However, arguments lacking an explicit conclusion create challenges for downstream processing tasks (Opitz et al, 2021;Alshomary et al, 2020;Gurcke et al, 2021). Thus, prior work has investigated approaches to make conclusions explicit.…”

“…Previous work has attempted to reconstruct a missing conclusion by identifying the "main target" in the premises (Alshomary et al, 2020). Other work has made use of pretrained sequenceto-sequence transformer language models finetuned on argumentative datasets (Syed et al, 2021;Opitz et al, 2021;Gurcke et al, 2021). However, the question of how to tailor a generated conclusion to a particular frame has not been systematically explored, a gap that we address with this paper.…”

Strategies for framing argumentative conclusion generation

“…Previous work has attempted to reconstruct a missing conclusion by identifying the "main target" in the premises (Alshomary et al, 2020). Other work has made use of pretrained sequenceto-sequence transformer language models finetuned on argumentative datasets (Syed et al, 2021;Opitz et al, 2021;Gurcke et al, 2021). However, the question of how to tailor a generated conclusion to a particular frame has not been systematically explored, a gap that we address with this paper.…”