How to Combine Visual Features with Tags to Improve Movie Recommendation Accuracy?

User Model User-Adap Inter

Dacrema

Constantin

et al. 2019

Self Cite

As of today, most movie recommendation services base their recommendations on collaborative filtering (CF) and/or content-based filtering (CBF) models that use metadata (e.g., genre or cast). In most video-on-demand and streaming services, however, new movies and TV series are continuously added. CF models are unable to make predictions in such a scenario, since the newly added videos lack interactions-a problem technically known as new item cold start (CS). Currently, the most common approach to this problem is to switch to a purely CBF method, usually by exploiting textual metadata. This approach is known to have lower accuracy than CF because it ignores useful collaborative information and relies on human-generated textual metadata, which are expensive to collect and often prone to errors. User-generated content, such as tags, can also be rare or absent in CS situations. In this paper, we introduce a new movie recommender system that addresses the new item problem in the movie domain by (i) integrating state-of-the-art audio and visual descriptors, which can be automatically extracted from video content and constitute what we call the movie genome; (ii) exploiting an effective data fusion method named canonical correlation analysis, which was successfully tested in our previous works Deldjoo et alto better exploit complementary information between different modalities; (iii) proposing a two-step hybrid approach which trains a CF model on warm items (items with interactions) and leverages the learned model on the movie genome to recommend cold items (items without interactions). Experimental validation is carried out using a system-centric study on a large-scale, real-world movie recommendation dataset both in an absolute cold start and in a cold to warm transition; and a user-centric online experiment measuring different subjective aspects, such as satisfaction and diversity. Results show the benefits of this approach compared to existing approaches.

Section: Contributions Of This Workmentioning

confidence: 99%

Section: Contributions Of This Workmentioning

confidence: 99%

Movie genome: alleviating new item cold start in movie recommendation

User Model User-Adap Inter

Dacrema

Constantin

et al. 2019

Self Cite

“…Motivated by the approach proposed in [14], we employed the fusion method based on Canonical Correlation Analysis (CCA) which exploits the low-level correlation between two set of visual features and learns a linear transformation that maximizes the pairwise correlation between two set of MPEG-7 and deep-learning networks visual features.…”

Section: Feature Fusionmentioning

confidence: 99%

Using visual features based on MPEG-7 and deep learning for movie recommendation

Int J Multimed Info Retr

Elahi

Quadrana

et al. 2018

Self Cite

Item features play an important role in movie recommender systems, where recommendations can be generated by using explicit or implicit preferences of users on traditional features (attributes) such as tag, genre, and cast. Typically, movie features are human-generated, either editorially (e.g., genre and cast) or by leveraging the wisdom of the crowd (e.g., tag), and as such, they are prone to noise and are expensive to collect. Moreover, these features are often rare or absent for new items, making it difficult or even impossible to provide good quality recommendations.In this paper, we show that user's preferences on movies can be better described in terms of the Mise-en-Scène features, i.e., the visual aspects of a movie that characterize design, aesthetics and style (e.g., colors, textures). We use both MPEG-7 visual descriptors and Deep Learning hidden layers as example of mise-en-scène features that can visually describe movies. Interestingly, mise-en-scène features can be computed automatically from video files or even from trailers, offering more flexibility in handling new items, avoiding the need for costly and error-prone human-based tagging, and providing good scalability.We have conducted a set of experiments on a large catalogue of 4K movies. Results show that recommendations based on mise-en-scène features consistently provide the best performance with respect to richer sets of more traditional features, such as genre and tag.

“…As for the last approach, the hybridization can be done at feature-level (e.g. in [12]) or ensemble-level. We chose the second approach for its ease of implementation.…”

Section: Procedurementioning

confidence: 99%

Exploring the Semantic Gap for Movie Recommendations

Elahi

Proceedings of the Eleventh ACM Conference on Recommender Systems

Moghaddam

et al. 2017

Self Cite

In the last years, there has been much a ention given to the semantic gap problem in multimedia retrieval systems. Much e ort has been devoted to bridge this gap by building tools for the extraction of high-level, semantics-based features from multimedia content, as low-level features are not considered useful because they deal primarily with representing the perceived content rather than the semantics of it.In this paper, we explore a di erent point of view by leveraging the gap between low-level and high-level features. We experiment with a recent approach for movie recommendation that extract lowlevel Mise-en-Scène features from multimedia content and combine it with high-level features provided by the wisdom of the crowd.To this end, we rst performed an o ine performance assessment by implementing a pure content-based recommender system with three di erent versions of the same algorithm, respectively based on (i) conventional movie a ributes, (ii) mise-en-scène features, and (iii) a hybrid method that interleaves recommendations based on movie a ributes and mise-en-scène features. In a second study, we designed an empirical study involving 100 subjects and collected data regarding the quality perceived by the users. Results from both studies show that the introduction of mise-en-scène features in conjunction with traditional movie a ributes improves both o ine and online quality of recommendations.