Novelty Detection in Social Media by Fusing Text and Image Into a Single Structure

Abstract

This work aims to propose an approach for detecting novelties, taking into account the temporal flow of data streams in social media. To this end, we present a completely new architecture for novelty detection. This new architecture entails three new contributions. First, we propose a new concept for novelty definition based on temporal windows. Second, we formulate an expression to determine the quality of a novelty. Third, we introduce a new approach to the fusion of heterogeneous data (image + text), using the COCO dataset and the MASK-RCNN convolutional neural network, which transforms image and text from social media into a single data format ready to be identified by machine learning algorithms. Since novelty detection is a task in which labeled samples are scarce or inexistent, unsupervised algorithms are used, and thus, the following baseline and state-of-the-art algorithms have been chosen: kNN, HBOS, FBagging, IForesting, and autoencoders. The new fusion approach is also compared to a state-of-the-art approach to outlier detection named AOM. Because of temporal particularities and the data types being fused, a new dataset was created, containing 27,494 tweets collected from Twitter. Our experiments show that data classification of social media using data fusion is superior to using only text or only images as input data.

New Media Public Relations Regulation Strategy Model Based on Generative Confrontation Network

“…Liu and He studied the challenges posed by new social media to government regulation of media from both theoretical and practical levels and introduced a theoretical model of government regulation of new media [16]. Amorim et al introduced public relations as a soft means into the regulation model by analyzing the "media access right" theory and the "media-source" relationship [17] [19]. Neijens summarized the social status and communication status of science communication in the new media era and summarized the communication methods and changes in the communication pattern in the new media era.…”

Survey on Novelty Detection using Machine Learning Techniques

“…Novelty detection methods covering a huge number of applications including fraud detection [1] , medical diagnosis [2], video surveillance [3], mobile robotics [4], social media [5] and among many others.…”

The Important Value of Social Media in the Construction of Korean National Image

“…Wang W, Ying s, Lyu J's purpose is to use a large number of online text data of social media to deconstruct all dimensions of the image of China's tourism boutique hotels, and strengthen the value creation of user generated content through social media [2]. Amorim m, bortoloti F D, ciarelli p m use coco data set and mask RCNN convolutional neural network to convert images and texts in social media into a single data format and identify them through machine learning algorithm, so as to introduce a new method to fuse heterogeneous data (image + text) [3]. Although the research results of social media are quite rich, there are still deficiencies in the research of social media in the construction of Korean national image.…”