Multimodal Deep Learning Framework for Image Popularity Prediction on Social Media

Abstract

Billions of photos are uploaded to the web daily through various types of social networks. Some of these images receive millions of views and become popular, whereas others remain completely unnoticed. This raises the problem of predicting image popularity on social media. The popularity of an image can be affected by several factors, such as visual content, aesthetic quality, user, post metadata, and time. Thus, considering all these factors is essential for accurately predicting image popularity. In addition, the efficiency of the predictive model also plays a crucial role. In this study, motivated by multimodal learning, which uses information from various modalities, and the current success of convolutional neural networks (CNNs) in various fields, we propose a deep learning model, called visual-social convolutional neural network (VSCNN), which predicts the popularity of a posted image by incorporating various types of visual and social features into a unified network model. VSCNN first learns to extract high-level representations from the input visual and social features by utilizing two individual CNNs. The outputs of these two networks are then fused into a joint network to estimate the popularity score in the output layer. We assess the performance of the proposed method by conducting extensive experiments on a dataset of approximately 432K images posted on Flickr. The simulation results demonstrate that the proposed VSCNN model significantly outperforms state-of-the-art models, with a relative improvement of greater than 2.33%, 7.59%, and 14.16% in terms of Spearman's Rho, mean absolute error, and mean squared error, respectively.

SmartNews: An Automatic Approach for Event Detection on Media Platforms

“…In addition to the contributions to this research subject; some surveys have reviewed this domain. Abousaleh, Cheng, Yu, & Tsao (2021) have reviewed dozens of research papers on popularity prediction considering multiple social media platforms. Also, some papers have addressed the problem of event detection in specified languages such as Arabic (Daoud & Daoud, 2020;Rafea & GabAllah, 2018), Chinese (Almerekhi, Hasanain, & Elsayed, 2016; Wang, Guo, & Wang, 2021), and more.…”

“…There are many research papers published on finding events in social media (e.g., see Chen, Xu, and Mao, 2019;Halimi and Ayday, 2020;Abousaleh, Cheng, Yu, and Tsao, 2021), but only a few of them focus on the problem of content selection of media events. Based on the above papers, among others, and the scarcity of published resources, in this paper, we address the problem of social media news event detection in an automated and smarter manner.…”

Optimizing Prediction of YouTube Video Popularity Using XGBoost

“…Researchers [35] present a deep learning model called visual-social convolutional neural network (VSCNN) that predicts the popularity of a posted image by mixing multiple types of visual and social data into a unified network model, which is motivated by multimodal learning, which incorporates input from many modalities and the current success of CNNs in numerous disciplines. In [36] the work conducted concerns what types of background should be use, whether and how to interpret contexts as a whole, and how to use prediction contexts effectively for FM music, Movie Lense, and Amazon Book.…”

Can We Foresee Landscape Interest? Maximum Entropy Applied to Social Media Photographs: A Case Study in Madrid

“…Numerous methods have unfolded to spatially define CES, as well as to characterize and to visualize them [7]. The more common tend to bring together quantitative and qualitative analysis, frequently combining land cover maps with image cluster analysis and automated image recognition [8][9][10][11][12][13].…”

“…Land 2022, 11, 715 2 of 13 Recently, a new line of research has been opened that consists of determining areas that can potentially provide CES from the current distribution of geotagged photos. For this purpose, distribution modeling software is used to identify degrees of significance of different environmental variables in relation to the photographs and to identify potential hotspot areas of CES.…”