JUST at SemEval-2020 Task 11: Detecting Propaganda Techniques Using BERT Pre-trained Model

Altiti, Ola; Abdullah, Malak; Obiedat, Rasha

doi:10.18653/v1/2020.semeval-1.229

Cited by 12 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their evaluation revealed that char n-gram produced the best results when combined with Nela. Likewise, other approaches such as Oliinyk et al (2020) and Altiti, Abdullah & Obiedat (2020) used TF-IDF, POS, lexicons-based, and word vectors for propaganda identification.…”

Section: Related Workmentioning

confidence: 99%

How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models

Malik

Imran

Mamdouh

2023

PeerJ Computer Science

View full text Add to dashboard Cite

Online propaganda is a mechanism to influence the opinions of social media users. It is a growing menace to public health, democratic institutions, and public society. The present study proposes a propaganda detection framework as a binary classification model based on a news repository. Several feature models are explored to develop a robust model such as part-of-speech, LIWC, word uni-gram, Embeddings from Language Models (ELMo), FastText, word2vec, latent semantic analysis (LSA), and char tri-gram feature models. Moreover, fine-tuning of the BERT is also performed. Three oversampling methods are investigated to handle the imbalance status of the Qprop dataset. SMOTE Edited Nearest Neighbors (ENN) presented the best results. The fine-tuning of BERT revealed that the BERT-320 sequence length is the best model. As a standalone model, the char tri-gram presented superior performance as compared to other features. The robust performance is observed against the combination of char tri-gram + BERT and char tri-gram + word2vec and they outperformed the two state-of-the-art baselines. In contrast to prior approaches, the addition of feature selection further improves the performance and achieved more than 97.60% recall, f1-score, and AUC on the dev and test part of the dataset. The findings of the present study can be used to organize news articles for various public news websites.

show abstract

Section: Related Workmentioning

confidence: 99%

How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models

Malik

Imran

Mamdouh

2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Therefore, it is practicable on a big scale and, at the very least, saves moderators from filtering through unwanted stuff. This automatic fact evaluation focuses on the article's material, claims, and statements rather than information such as the source or rate of dissemination [53][54][55]. Furthermore, the ClaimRank technique identifies claims needing to be verified and refers to fact-checking websites that utilize manual or automated techniques to verify claims [56].…”

Section: Related Workmentioning

confidence: 99%

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

et al. 2023

View full text Add to dashboard Cite

Social media, fake news, and different propaganda strategies have all contributed to an increase in misinformation online during the past ten years. As a result of the scarcity of high-quality data, the present datasets cannot be used to train a deep-learning model, making it impossible to establish an identification. We used a natural language processing approach to the issue in order to create a system that uses deep learning to automatically identify propaganda in news items. To assist the scholarly community in identifying propaganda in text news, this study suggested the propaganda texts (ProText) library. Truthfulness labels are assigned to ProText repositories after being manually and automatically verified with fact-checking methods. Additionally, this study proposed using a fine-tuned Robustly Optimized BERT Pre-training Approach (RoBERTa) and word embedding using multi-label multi-class text classification. Through experimentation and comparative research analysis, we address critical issues and collaborate to discover answers. We achieved an evaluation performance accuracy of 90%, 75%, 68%, and 65% on ProText, PTC, TSHP-17, and Qprop, respectively. The big-data method, particularly with deep-learning models, can assist us in filling out unsatisfactory big data in a novel text classification strategy. We urge collaboration to inspire researchers to acquire, exchange datasets, and develop a standard aimed at organizing, labeling, and fact-checking.

show abstract

“…As an example, the CN-HIT-IT.NLP team and ECNU-SenseMaker (Zhao et al, 2020) both used a variant of K-BERT (Liu et al, 2020a) with additional data; the former injects relevant triples from ConceptNet to the language model, while the later also uses ConceptNet's unstructured text to pre-train the language model. Other systems relied on ensemble models consisting of different language models such as RoBERTa and XLNet (Liu, 2020;Altiti et al, 2020).…”

Section: Arxiv:221003378v1 [Cscl] 7 Oct 2022 2 Backgroundmentioning

confidence: 99%

UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation

Sarhan¹,

Mosteiro²,

Spruit³

2022

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)

View full text Add to dashboard Cite

This paper presents our strategy to address the SemEval-2022 Task 3 PreTENS: Presupposed Taxonomies Evaluating Neural Network Semantics. The goal of the task is to identify if a sentence is deemed acceptable or not, depending on the taxonomic relationship that holds between a noun pair contained in the sentence. For sub-task 1-binary classification-we propose an effective way to enhance the robustness and the generalizability of language models for better classification on this downstream task. We design a two-stage fine-tuning procedure on the ELECTRA language model using data augmentation techniques. Rigorous experiments are carried out using multi-task learning and data-enriched fine-tuning. Experimental results demonstrate that our proposed model, UU-Tax, is indeed able to generalize well for our downstream task. For sub-task 2regression-we propose a simple classifier that trains on features obtained from Universal Sentence Encoder (USE). In addition to describing the submitted systems, we discuss other experiments that employ pre-trained language models and data augmentation techniques. For both sub-tasks, we perform error analysis to further understand the behaviour of the proposed models. We achieved a global F1 Binary score of 91.25% in sub-task 1 and a rho score of 0.221 in sub-task 2. 1

show abstract

JUST at SemEval-2020 Task 11: Detecting Propaganda Techniques Using BERT Pre-trained Model

Cited by 12 publications

References 21 publications

How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models

How to detect propaganda from social media? Exploitation of semantic and fine-tuned language models

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation

Contact Info

Product

Resources

About