“Bend the truth”: Benchmark dataset for fake news detection in Urdu language and its evaluation

Amjad, Maaz; Sidorov, Grigori; Zhila, Alisa; Gómez-Adorno, Helena; Voronkov, Ilia M.; Gelbukh, Alexander

doi:10.3233/jifs-179905

Cited by 46 publications

(58 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More recently, Wang et al (2020) collected a Weibo dataset containing 7,300 news articles in Chinese across eight domains (health, economic, technology, entertainment, society, military, political, and education). Similarly, Amjad et al (2020) proposed a benchmark dataset in the Urdu language that contains 900 news articles in 5 different domains (business, health, showbiz, sports, and technology). Therefore, to overcome the limitations on the diversity of fact-checking, future datasets should be collected from a variety of news domains.…”

Section: Survey Methodologymentioning

confidence: 99%

Fake news detection: a survey of evaluation datasets

D’Ulizia¹,

Caschera²,

Ferri³

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

Fake news detection has gained increasing importance among the research community due to the widespread diffusion of fake news through media platforms. Many dataset have been released in the last few years, aiming to assess the performance of fake news detection methods. In this survey, we systematically review twenty-seven popular datasets for fake news detection by providing insights into the characteristics of each dataset and comparative analysis among them. A fake news detection datasets characterization composed of eleven characteristics extracted from the surveyed datasets is provided, along with a set of requirements for comparing and building new datasets. Due to the ongoing interest in this research topic, the results of the analysis are valuable to many researchers to guide the selection or definition of suitable datasets for evaluating their fake news detection methods.

show abstract

Section: Survey Methodologymentioning

confidence: 99%

Fake news detection: a survey of evaluation datasets

D’Ulizia¹,

Caschera²,

Ferri³

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Existe un predominio de los trabajos que proponen sistemas o métodos completos para la detección de contenidos desinformativos (n=74; 77,08%), frente a la menor cantidad de estudios sobre bases de datos (n=11; 11,45%) y algoritmos específicos (n=9; 9,37%) (tabla 5). En el caso de las bases de datos, son mayoritarias las realizadas en lengua inglesa con el fin de entrenar algoritmos utilizados en modelos de análisis lingüístico en este idioma; si bien existen aproximaciones centradas en otros, como el portugués (Silva, Santos, Almeida y Pardo, 2020), el español (Posadas-Durán, Gómez-Adorno, Sidorov y Moreno-Escobar, 2019) y lenguas minoritarias como el urdu (Amjad et al 2020).…”

Section: Años De Publicación Países Propuestas Y Modelos Algorítmicosunclassified

Modelos algorítmicos y fact-checking automatizado. Revisión sistemática de la literatura

García-Marín

2022

Doc. cienc. inf.

View full text Add to dashboard Cite

El fact-checking automatizado consiste en la comprobación automática de la veracidad de una información aplicando las tecnologías de inteligencia artificial existentes para clasificarla en alguna de las categorías comúnmente usadas por los fact-checkers humanos (verdadero, engañoso, falso, etc.). Este trabajo presenta el primer análisis bibliométrico en castellano -de tipo cuantitativo- sobre la evolución y los países de procedencia de la investigación sobre esta práctica. Asimismo, pretende analizar el nivel de precisión de las soluciones algorítmicas y el impacto de los trabajos publicados, utilizando para ello tratamientos estadísticos descriptivos e inferenciales (pruebas de chi cuadrado y test de Kruskal-Wallis). De acuerdo con nuestros resultados, en los últimos tres años se concentra el mayor volumen de aportaciones, que proceden mayoritariamente de la región asiática y Estados Unidos. Predominan los trabajos que proponen métodos o sistemas algorítmicos integrados. Son mayoritarios los estudios sobre modelos lingüísticos, que presentan aún varias limitaciones y una efectividad inferior a la media. Se observa una reducida atención hacia los modelos basados en el análisis de imágenes, y resulta prácticamente nula la presencia de algoritmos de detección de audios falsos. En línea con trabajos anteriores, nuestro estudio concluye que no existen diferencias estadísticamente significativas en el nivel de precisión de los diversos modelos algorítmicos propuestos, a pesar de sus diferentes grados de complejidad técnica.

show abstract

“…To detect fake news, they performed SVM, LR, RF, and boosting on bag of words (BOW), POS tags, and n-grams features sets of their datasets and find character 4-grams without removing the stop words with the Boosting algorithm has the best performance in accuracy. Amjad et al [5] proposed a new Urdu language corpus: "Bend The Truth" for fake news detection which contains 900 news articles, 500 annotated as real and 400 labeled as fake. Their text representation feature sets include the combination of word n-grams, character n-grams, functional word n-grams (n ranging from 1 to 6) with a variety of feature weighting schemes including binary values, normalized frequency, log-entropy weighting, raw frequency, relative frequency, and TF-IDF.…”

Section: Literature Reviewmentioning

confidence: 99%

A Novel Stacking Approach for Accurate Detection of Fake News

et al. 2021

View full text Add to dashboard Cite

With the increasing popularity of social media, people has changed the way they access news. News online has become the major source of information for people. However, much information appearing on the Internet is dubious and even intended to mislead. Some fake news are so similar to the real ones that it is difficult for human to identify them. Therefore, automated fake news detection tools like machine learning and deep learning models have become an essential requirement. In this paper, we evaluated the performance of five machine learning models and three deep learning models on two fake and real news datasets of different size with hold out cross validation. We also used term frequency, term frequencyinverse document frequency and embedding techniques to obtain text representation for machine learning and deep learning models respectively. To evaluate models' performance, we used accuracy, precision, recall and F1-score as the evaluation metrics and a corrected version of McNemar's test to determine if models' performance is significantly different. Then, we proposed our novel stacking model which achieved testing accuracy of 99.94% and 96.05 % respectively on the ISOT dataset and KDnugget dataset. Furthermore, the performance of our proposed method is high as compared to baseline methods. Thus, we highly recommend it for fake news detection.

show abstract

“Bend the truth”: Benchmark dataset for fake news detection in Urdu language and its evaluation

Cited by 46 publications

References 6 publications

Fake news detection: a survey of evaluation datasets

Fake news detection: a survey of evaluation datasets

Modelos algorítmicos y fact-checking automatizado. Revisión sistemática de la literatura

A Novel Stacking Approach for Accurate Detection of Fake News

Contact Info

Product

Resources

About