A Comprehensive Survey on Multimodal Recommender Systems: Taxonomy, Evaluation, and Future Directions

Abstract

Recommendation systems have become popular and effective tools to help users discover their interesting items by modeling the user preference and item property based on implicit interactions (e.g., purchasing and clicking). Humans perceive the world by processing the modality signals (e.g., audio, text and image), which inspired researchers to build a recommender system that can understand and interpret data from different modalities. Those models could capture the hidden relations between different modalities and possibly recover the complementary information which can not be captured by a uni-modal approach and implicit interactions. The goal of this survey is to provide a comprehensive review of the recent research efforts on the multimodal recommendation. Specifically, it shows a clear pipeline with commonly used techniques in each step and classifies the models by the methods used. Additionally, a code framework has been designed that helps researchers new in this area to understand the principles and techniques, and easily runs the SOTA models. Our framework is located at: https://github.com/enoche/MMRec.

Enhancing Dyadic Relations with Homogeneous Graphs for Multimodal Recommendation

“…BM3 [34] bootstraps latent representations of both ID embeddings and multimodal features using a contrastive view generator and designs three contrastive objective functions to optimize representations for effective and efficient recommendations. For an in-depth exploration of multimodal recommender systems, we recommend consulting the comprehensive survey conducted by [29].…”