A Hierarchical Attention Model for Social Contextual Image Recommendation

Abstract

Image based social networks are among the most popular social networking services in recent years. With tremendous images uploaded everyday, understanding users' preferences on user-generated images and making recommendations have become an urgent need. In fact, many hybrid models have been proposed to fuse various kinds of side information (e.g., image visual representation, social network) and user-item historical behavior for enhancing recommendation performance. However, due to the unique characteristics of the user generated images in social image platforms, the previous studies failed to capture the complex aspects that influence users' preferences in a unified framework. Moreover, most of these hybrid models relied on predefined weights in combining different kinds of information, which usually resulted in sub-optimal recommendation performance. To this end, in this paper, we develop a hierarchical attention model for social contextual image recommendation. In addition to basic latent user interest modeling in the popular matrix factorization based recommendation, we identify three key aspects (i.e., upload history, social influence, and owner admiration) that affect each user's latent preferences, where each aspect summarizes a contextual factor from the complex relationships between users and images. After that, we design a hierarchical attention network that naturally mirrors the hierarchical relationship (elements in each aspects level, and the aspect level) of users' latent interests with the identified key aspects. Specifically, by taking embeddings from state-of-the-art deep learning models that are tailored for each kind of data, the hierarchical attention network could learn to attend differently to more or less content. Finally, extensive experimental results on real-world datasets clearly show the superiority of our proposed model. !

Fine-Grained Similarity Measurement between Educational Videos and Exercises

“…Along this line, methods for modeling text pairs [23,42] can be applied to learn the similarity of video-exercise pairs based on their textual materials. However, the visual information can be used to enhance the sentence semantic understanding [46] and the recommendation performance [38]. Therefore, in VENet, we exploit the visual data as supplement information to the closed captions to accurately understand and represent the video.…”

“…For example, YouEDU [1] automatically recommends related video snippets for the forum posts based on the cosine similarity between the closed captions and the post description. However, the visual information can be used to enhance the sentence semantic understanding [46] and the recommendation performance [38]. Besides, most of them are heavily dependent on the scarce labeled data on segments.…”

Learning from visual content and style: an image-enhanced recommendation model

“…Given the rating matrix R and the corresponding images of items, we identify that a users' preferences are influenced by three aspects. Specifically, the ratings matrix R from each user's feedback is well recognized as a latent and an important factor in the image recommendation (He and McAuley 2016;Wu et al 2019). When a user sees a new image, it's natural to pay attention to the style and content of the image.…”

“…Visual signals are playing more and more important roles for users' daily life (Wu et al 2019). Especially in modern fast-paced society, users are reluctant to spend much time to read literal content, but prefer to browser visual images (Gelli et al 2018).…”

“…Besides the image content modeling for recommendation, image styles also play an important role for users' visual experience. When browsing images, users' preferences are not only decided by "what is the content of this information", but also "does the image style matches my taste" (Wu et al 2019). While visual content features are empirically effective for image based recommendation, the visual styles are largely ignored in the image recommendation research community.…”