LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation

Abstract

This paper presents one of the top systems for the SemEval-2021 task 6 (Dimitrov et al., 2021, "detection of persuasion techniques in text and images". The proposed system, Le-Cun, targets subtask-1 for detecting propaganda techniques based on the textual content of a meme. We have used an external dataset from a previous relevant SemEval competition (Martino et al., 2020). We also have articulated another dataset using dataaugmentation techniques. The final proposed model consisted of 5 ensemble transformers (four RoBERTa models and one DeBERTa), each trained on either a different dataset or preprocessing. Apparently, ensembling models trained on different datasets improve performance more than ensembling models trained on the same dataset/preprocessing. Also, it is obvious, fine-tuning the model on the Competition dataset after training it for a few epochs on the other datasets would improve the f1micro up to 0.1 additional scores. The final model achieved an f1-micro score of 0.512 on the test dataset and an f1-micro of 0.647 on the development dataset.

Detecting Persuasion Attempts on Social Networks: Unearthing the Potential of Loss Functions and Text Pre-Processing in Imbalanced Data Settings

“…By looking at Table 1, we can spot a clear imbalance between classes, with Loaded Language, Name Calling and Smears appearing on over 25% of the samples while Bandwagon, Irrelevant Data, Confusion, Reductio ad Hitlerum and Repetition are present in less than 2% of the data. This imbalance explains why some models are not able to predict labels such as Bandwagon, Red Herring and Obfuscation, Intentional vagueness, Confusion [6,8].…”

“…The SemEval task we proposed to solve [3] was tackled by many groups with different approaches, although most of them revolved around the use of transformers. Some of those groups [6,7] tried to mitigate the scarcity of data and the class imbalance by augmenting the dataset using different techniques. The results were far from positive, when techniques such as random swap, synonym replacement, random deletion and random insertion were used to augment the dataset, the model's performance decreased.…”

Legend at ArAIEval Shared Task: Persuasion Technique Detection using a Language-Agnostic Text Representation Model

“…A significant amount of research has been conducted on the detection of persuasion techniques in text (Hasanain et al, 2023;Modzelewski et al, 2023;Hossain et al, 2021;Sadeghi et al, 2023;Abujaber et al, 2021). Researchers have explored binary and multilabel approaches to detecting persuasion techniques.…”

“…In an era defined by the rapid dissemination of information through digital channels, the task of recognizing persuasion techniques in text is now more crucial than ever (Hasanain et al, 2023;Hossain et al, 2021;Gupta et al, 2021;Sadeghi et al, 2023;Alam et al, 2022;Abujaber et al, 2021). The advent of the Internet and social media has created new avenues for influence and manipulation (Dholakia et al, 2023;Ruffo et al, 2023;Botes, 2023).…”

SarcasmDet at SemEval-2022 Task 6: Detecting Sarcasm using Pre-trained Transformers in English and Arabic Languages

“…As a result, machines became able to understand the underlying meanings of words and not just rely on keywords. Moreover, it helps reveal forms of speech, such as emotion analysis, humor, ridicule, abuse, etc (Abujaber et al, 2021;Qarqaz et al, 2021).…”