Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media

Abstract

In social networks, users can easily share information and express their opinions. Given the huge amount of data posted by many users, it is difficult to search for relevant information. In addition to individual posts, it would be useful if we can recommend groups of people with similar interests. Past studies on user preference learning focused on single-modal features such as review contents or demographic information of users. However, such information is usually not easy to obtain in most social media without explicit user feedback. In this paper, we propose a multimodal feature fusion approach to implicit user preference prediction which combines text and image features from user posts for recommending similar users in social media. First, we use the convolutional neural network (CNN) and TextCNN models to extract image and text features, respectively. Then, these features are combined using early and late fusion methods as a representation of user preferences. Lastly, a list of users with the most similar preferences are recommended. The experimental results on real-world Instagram data show that the best performance can be achieved when we apply late fusion of individual classification results for images and texts, with the best average top-k accuracy of 0.491. This validates the effectiveness of utilizing deep learning methods for fusing multimodal features to represent social user preferences. Further investigation is needed to verify the performance in different types of social media.

Adaptive Sensitive Information Recognition Based on Multimodal Information Inference in Social Networks

“…Chen et al [49] adopted a dual-channel multihop inference mechanism to reason and fuse image features and text features to achieve cross-modal information interaction. Besides, Wang et al [50] applied multimodal fusion to similarity user recommendation system and proposed an implicit user preference prediction method with multimodal feature fusion. Combining text and image features in user posts, the image and text features are extracted using convolutional neural network (CNN) and text CNN models, respectively, and then these features are combined as a representation of user preferences using early and late fusion methods.…”

Modeling and Applying Implicit Dormant Features for Recommendation via Clustering and Deep Factorization

“…For example, several businesses, including Artsy, Netflix, and Pandora Internet Radio [ 15 ], have created unique clustering-based recommendation systems Art Genome Project, Micro-Genres of Movies, and Music Genome Project, respectively. In addition, many research works [ 16 , 17 , 18 , 19 ] have already been carried out on clustering and learning representative features of users in terms of similarity, which are important for modeling recommender system. Without regard to whether the length of the music list consumed is short or not, ref.…”

Deep learning application detecting SARS-CoV-2 key enzymes inhibitors

“…As a result, various fields bloomed after supporting it with AI such as computer vision [ 1 ] where the computers are trained to identify objects from images and videos. Various deep learning (DL) applications appeared, like smart healthcare [ 2 ], gaming [ 3 ], and social media where the AI helps make an accurate prediction about user preferences [ 4 ]. Additionally, its application to health systems where a computer can identify the presence of diseases from MRI and CT images such as breast cancer, COVID-19 [ 5 ], etc.…”