Background music recommendation based on latent factors and moods

Liu, Chien‐Liang; Chen, Ying Chuan

doi:10.1016/j.knosys.2018.07.001

Cited by 33 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Emotion tag-based work uses the relatedness between visual tags and music mood tags [5,6,9]. Moreover, emotion model [4] has also been used to match video-music pairs [1,3]. These emotion tags for the video, however, were manually annotated from crowdsourcing, which is high in labor and time costs.…”

Section: Audiovisual Cross-modal Matchingmentioning

confidence: 99%

“…Uploaders can record or make videos and select appropriate background music by themselves. However, the huge music pool makes manual selection time-consuming and labor-intensive, so it is necessary to recommend suitable background music for micro-videos [1,2].…”

Section: Introductionmentioning

confidence: 99%

“…Existing work on recommending background music to videos mainly focuses on designing models to achieve the matching of video and background music. Chen et al [1] and Sasaki et al [3] relied on the emotion model in [4] to divide music and videos into different emotional regions for matching. Chao et al [5] leveraged the mood tags of music and videos to calculate the relatedness graph for recommending cross-media content.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Debiased Cross-modal Matching for Content-based Micro-video Background Music Recommendation

Yi¹,

Chen²

2022

Preprint

View full text Add to dashboard Cite

Micro-video background music recommendation is a complicated task where the matching degree between videos and uploader-selected background music is a major issue. However, the selection of the user-generated content (UGC) is biased caused by knowledge limitations and historical preferences among music of each uploader. In this paper, we propose a Debiased Cross-Modal (DebCM) matching model to alleviate the influence of such selection bias. Specifically, we design a teacher-student network to utilize the matching of segments of music videos, which is professional-generated content (PGC) with specialized music-matching techniques, to better alleviate the bias caused by insufficient knowledge of users. The PGC data is captured by a teacher network to guide the matching of uploader-selected UGC data of the student network by KL-based knowledge transfer. In addition, uploaders' personal preferences of music genres are identified as confounders that spuriously correlate music embeddings and background music selections, resulting in the learned recommender system to over-recommend music from the majority groups. To resolve such confounders in the UGC data of the student network, backdoor adjustment is utilized to deconfound the spurious correlation between music embeddings and prediction scores. We further utilize Monte Carlo (MC) estimator with batch-level average as the approximations to avoid integrating the entire confounder space calculated by the adjustment. Extensive experiments on the TT-150k-genre dataset demonstrate the effectiveness of the proposed method towards the selection bias. The code is publicly available on: https://github.com/jing-1/DebCM.

show abstract

Section: Audiovisual Cross-modal Matchingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Debiased Cross-modal Matching for Content-based Micro-video Background Music Recommendation

Yi¹,

Chen²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The recommendations are generated by finding the nearest neighbours of the projected video features from the database of projected music features. Liu and Chen [79] proposed to extract low-level features for both videos and musics, paying special attention to the emotional undertones of the video contents and the musics. The relevance scores are then calculated for recommendation, based on a custom designed emotion-aware scoring function.…”

Section: Music Recommendation and Generationmentioning

confidence: 99%

AI-Empowered Persuasive Video Generation: A Survey

Liu¹,

Han²

2021

Preprint

View full text Add to dashboard Cite

Promotional videos are rapidly becoming a popular medium for persuading people to change their behaviours in many settings (e.g., online shopping, social enterprise initiatives). Today, such videos are often produced by professionals, which is a time-, labour-and cost-intensive undertaking. In order to produce such contents to support a large applications (e.g., e-commerce), the field of artificial intelligence (AI)-empowered persuasive video generation (AIPVG) has gained traction in recent years. This field is interdisciplinary in nature, which makes it challenging for new researchers to grasp. Currently, there is no comprehensive survey of AIPVG available. In this paper, we bridge this gap by reviewing key AI techniques that can be utilized to automatically generate persuasive videos. We offer a first-of-its-kind taxonomy which divides AIPVG into three major steps: 1) visual material understanding, which extracts information from the visual materials (VMs) relevant to the target of promotion; 2) visual storyline generation, which shortlists and arranges high-quality VMs into a sequence in order to compose a storyline with persuasive power; and 3) post-production, which involves background music generation and still image animation to enhance viewing experience. We also introduce the evaluation metrics and datasets commonly adopted in the field of AIPVG. We analyze the advantages and disadvantages of the existing works belonging to the above-mentioned steps, and discuss interesting potential future research directions.

show abstract

“…The contextual recommendation of songs is based on those emotional states. A mood based recommendation of music for videos is proposed in [69]. Both videos and music are projected onto a latent emotional space and a latent factor model is used to find the relevance between videos and music.…”

Section: Temporal Contextmentioning

confidence: 99%

Time-Aware Music Recommender Systems: Modeling the Evolution of Implicit User Preferences and User Listening Habits in A Collaborative Filtering Approach

2020

View full text Add to dashboard Cite

Online streaming services have become the most popular way of listening to music. The majority of these services are endowed with recommendation mechanisms that help users to discover songs and artists that may interest them from the vast amount of music available. However, many are not reliable as they may not take into account contextual aspects or the ever-evolving user behavior. Therefore, it is necessary to develop systems that consider these aspects. In the field of music, time is one of the most important factors influencing user preferences and managing its effects, and is the motivation behind the work presented in this paper. Here, the temporal information regarding when songs are played is examined. The purpose is to model both the evolution of user preferences in the form of evolving implicit ratings and user listening behavior. In the collaborative filtering method proposed in this work, daily listening habits are captured in order to characterize users and provide them with more reliable recommendations. The results of the validation prove that this approach outperforms other methods in generating both context-aware and context-free recommendations.

show abstract

Background music recommendation based on latent factors and moods

Cited by 33 publications

References 27 publications

Debiased Cross-modal Matching for Content-based Micro-video Background Music Recommendation

Debiased Cross-modal Matching for Content-based Micro-video Background Music Recommendation

AI-Empowered Persuasive Video Generation: A Survey

Time-Aware Music Recommender Systems: Modeling the Evolution of Implicit User Preferences and User Listening Habits in A Collaborative Filtering Approach

Contact Info

Product

Resources

About