Detecting fake news by exploring the consistency of multimodal data

Xue, Junxiao; Wang, Yabo; Tian, Yichen; Li, Yafei; Shi, Lei; Wei, Lin

doi:10.1016/j.ipm.2021.102610

Cited by 137 publications

(44 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Since it is difficult to find both pertinent and pristine images to match these fictions, fake news generators sometimes use manipulated images to support non-factual scenarios. Researchers refer to this cue as the similarity relationship between text and image (Zhou, Wu, and Zafarani 2020;Giachanou, Zhang, and Rosso 2020;Xue et al 2021) which could be captured with a variety of similarity measuring techniques such as cosine similarity between the title and image tags embeddings (Zhou, Wu, and Zafarani 2020;Giachanou, Zhang, and Rosso 2020) or similarity measure architectures (Xue et al 2021).…”

Section: Multi-modal Features and Cluesmentioning

confidence: 99%

“…(Singhal et al 2021) develop an inter-modality discordance based fake news detection which learns discriminating features and employs a modified version of contrastive loss that explores the inter-modality discordance. Xue et al (Xue et al 2021), propose a Multimodal Consistency Neural Network (MCNN) which utilizes a similarity measurement module that measures the similarity of multi-modal data to detect the possible mismatches between the image and text. Lastly, Biamby et al (Biamby et al 2021),leverage a Contrastive Language-Image Pre-Training (CLIP) model (Radford et al 2021), to jointly learn image/text representation to detect Image-Text incon-sistencies in Tweets.…”

Section: Generative Architecturesmentioning

confidence: 99%

See 1 more Smart Citation

Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

Abdali¹

2022

Preprint

View full text Add to dashboard Cite

As social media platforms are evolving from text-based forums into multi-modal environments, the nature of misinformation in social media is also changing accordingly. Taking advantage of the fact that visual modalities such as images and videos are more favorable and attractive to the users, and textual contents are sometimes skimmed carelessly, misinformation spreaders have recently targeted contextual correlations between modalities e.g., text and image. Thus, many research efforts have been put into development of automatic techniques for detecting possible cross-modal discordances in web-based media. In this work, we aim to analyze, categorize and identify existing approaches in addition to challenges and shortcomings they face in order to unearth new opportunities in furthering the research in the field of multi-modal misinformation detection.

show abstract

Section: Multi-modal Features and Cluesmentioning

confidence: 99%

Section: Generative Architecturesmentioning

confidence: 99%

Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

Abdali¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead of obtaining in-the-wild knowledge, recent works leveraged entity background information obtained from knowledge graphs (Cui, Seo, Tabar, Ma, Wang and Lee, 2020;Zhang, Fang, Qian and Xu, 2019;Hu, Yang, Zhang, Zhong, Tang, Shi, Duan and Zhou, 2021). For multi-modal scenarios, entity knowledge is important to bridge the text-image semantics (Xue, Wang, Tian, Li, Shi and Wei, 2021;Qi, Cao and Sheng, 2021b;Qi, Cao, Li, Liu, Sheng, Mi, He, Lv, Guo and Yu, 2021a;Li, Sun, Yu, Tian, Yao and Xu, 2021). These methods could provide accurate and explainable evidence, but have the issue of source credibility and scalability.…”

Section: False News Detectionmentioning

confidence: 99%

Characterizing Multi-Domain False News and Underlying User Effects on Chinese Weibo

Sheng,

Cao,

Bernard

et al. 2022

Preprint

View full text Add to dashboard Cite

False news that spreads on social media has proliferated over the past years and has led to multi-aspect threats in the real world. While there are studies of false news on specific domains (like politics or health care), little work is found comparing false news across domains. In this article, we investigate false news across nine domains on Weibo, the largest Twitterlike social media platform in China, from 2009 to 2019. The newly collected data comprise 44,728 posts in the nine domains, published by 40,215 users, and reposted over 3.4 million times. Based on the distributions and spreads of the multi-domain dataset, we observe that false news in domains that are close to daily life like health and medicine generated more posts but diffused less effectively than those in other domains like politics, and that political false news had the most effective capacity for diffusion. The widely diffused false news posts on Weibo were associated strongly with certain types of users -by gender, age, etc. Further, these posts provoked strong emotions in the reposts and diffused further with the active engagement of false-news starters. Our findings have the potential to help design false news detection systems in suspicious news discovery, veracity prediction, and display and explanation. The comparison of the findings on Weibo with those of existing work demonstrates nuanced patterns, suggesting the need for more research on data from diverse platforms, countries, or languages to tackle the global issue of false news. The code and new anonymized dataset are available at https://github.com/ICTMCG/Characterizing-Weibo-Multi-Domain-False-News.

show abstract

“…However, these studies do not consider consistencies between multi-modal information as our work. While both SAFE (Zhou et al, 2020) and MCNN (Xue et al, 2021) consider relevance between textual and visual information, our work differs from theirs in that we distinguish modal-unique from modal-shared information, and also model inconsistencies between content and external knowledge.…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, it also provides a great opportunity to identify rumors. Xue et al (2021) shows that, in order to catch eyes of public, rumors tend to use theatrical, comical and attractive images that are irrelevant to the post content. In general, it is often difficult to find pertinent and non-manipulated images to match fictional events, thus posts with mismatched textual and visual information are more likely to be fake (Zhou et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Inconsistency Matters: A Knowledge-guided Dual-inconsistency Network for Multi-modal Rumor Detection

Sun¹,

Zhang²,

Ma³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Rumor spreaders are increasingly utilizing multimedia content to attract the attention and trust of news consumers. Though a set of rumor detection models have exploited the multimodal data, they seldom consider the inconsistent relationships among images and texts. Moreover, they also fail to find a powerful way to spot the inconsistency information among the post contents and background knowledge. Motivated by the intuition that rumors are more likely to have inconsistency information in semantics, a novel Knowledge-guided Dualinconsistency network is proposed to detect rumors with multimedia contents. It can capture the inconsistent semantics at the cross-modal level and the content-knowledge level in one unified framework. Extensive experiments on two public real-world datasets demonstrate that our proposal can outperform the state-ofthe-art baselines.

show abstract

Detecting fake news by exploring the consistency of multimodal data

Cited by 137 publications

References 37 publications

Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

Characterizing Multi-Domain False News and Underlying User Effects on Chinese Weibo

Inconsistency Matters: A Knowledge-guided Dual-inconsistency Network for Multi-modal Rumor Detection

Contact Info

Product

Resources

About