MSRD: Multimodal Web Rumor Detection Method

Abstract

Multimodal web rumors, which combine images and text, are confusing and can be inflammatory, and therefore can be harmful to national security and social stability. Currently, web rumor detection fully considers text content but ignores image content, including text embedded in images. This paper proposes a multimodal web rumor detection method based on a deep neural network considering images, image-embedded text, and text content. This method uses a VGG-19 network to extract image content features, DenseNet to extract embedded text content, and an LSTM (Long Short-term Memory) network to extract text content features. After concatenation with image features, the mean and variance vectors of the image and text shared representations are obtained through a completely connected layer, and random variables sampled from a Gaussian distribution are used to form a reparameterized multimodal feature as the input of the rumor detector. Experiments show that the accuracy of this method is 68.5% and 79.4% on Twitter and Weibo, respectively.

Multilevel Attention Residual Neural Network for Multimodal Online Social Network Rumor Detection

“…Dhruv et al [23] then constrained the fused multimodal vectors through an automatic encoder to better learn the joint representation. Liu et al [24] made full use of the text information contained in the image and improved the detection performance of the model by extracting hidden texts from the image.…”

“…In this model [24], First, the text in the image is extracted, and then it is connected with the text content in the sample. Finally, the image and the connected text are fused and classified at the feature level.…”

Research on Rumor Detection Based on a Graph Attention Network With Temporal Features

“…These methods demonstrate the importance of temporal features in rumor detection by extracting temporal relationships between tweets or keywords. Gao et al (2020) used task-specific features based on bidirectional language models to learn contextual embedded textual information and event sequence information; Liu et al (2020) used deep learning to extract the text features of tweets, image features and text information in images. However, these kinds of methods only stay at the most basic surface features and cannot extract high-level, abstract global features of rumors.…”