Learning Audio Embeddings with User Listening Data for Content-Based Music Recommendation

Chen, Ke; Liang, Beici; Ma, Xiaoshuan; Gu, Minwei

doi:10.1109/icassp39728.2021.9414458

Cited by 28 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…And more sources can be separated via one model. In future work, since audio embeddings have been widely used in other audio tasks such as music recommendation (Chen et al 2021) andmusic generation (2019;2021;, we expect to use these audio embeddings as source queries to see if they can capture different audio features and lead to better separation performance.…”

Section: Discussionmentioning

confidence: 99%

Zero-Shot Audio Source Separation through Query-Based Learning from Weakly-Labeled Data

Chen

Du²,

Zhu³

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

Deep learning techniques for separating audio into different sound sources face several challenges. Standard architectures require training separate models for different types of audio sources. Although some universal separators employ a single model to target multiple sources, they have difficulty generalizing to unseen sources. In this paper, we propose a three-component pipeline to train a universal audio source separator from a large, but weakly-labeled dataset: AudioSet. First, we propose a transformer-based sound event detection system for processing weakly-labeled training data. Second, we devise a query-based audio separation model that leverages this data for model training. Third, we design a latent embedding processor to encode queries that specify audio targets for separation, allowing for zero-shot generalization. Our approach uses a single model for source separation of multiple sound types, and relies solely on weakly-labeled data for training. In addition, the proposed audio separator can be used in a zero-shot setting, learning to separate types of audio sources that were never seen in training. To evaluate the separation performance, we test our model on MUSDB18, while training on the disjoint AudioSet. We further verify the zero-shot performance by conducting another experiment on audio source types that are held-out from training. The model achieves comparable Source-to-Distortion Ratio (SDR) performance to current supervised models in both cases.

show abstract

Section: Discussionmentioning

confidence: 99%

Zero-Shot Audio Source Separation through Query-Based Learning from Weakly-Labeled Data

Chen

Du²,

Zhu³

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several works have utilized VAE models for exploring audio representations, including VAE models for finding disentangled audio representations (Luo, Agres, & Herremans, 2019) and VAE models for modeling audio containing speech (Hsu, Zhang, & Glass, 2017). Apart from VAE, simple convolutional models are used in different music tasks such as music recommendation (Chen, Liang, Ma, & Gu, 2021) and source separation (Chen, Du, et al, 2021). In our experiments for audio representations, we chose to use a convolutional VAE to map the short-time Fourier transform (STFT) representation of audio into a lower dimensional latent representation.…”

Section: Vae For Audiomentioning

confidence: 99%

Deep Music Information Dynamics Novel Framework for Reduced Neural-Network Music Representation with Applications to Midi and Audio Analysis and Improvisation

Dubnov¹,

Chen²,

Huang³

2022

Journal of Creative Music Systems

Self Cite

View full text Add to dashboard Cite

Generative musical models often comprise of multiple levels of structure, presuming that the process of composition moves between background to foreground, or between generating musical surface and some deeper and reduced representation that governs hidden or latent dimensions of music.  In this paper we are using a recently proposed framework called Deep Musical Information Dynamics (DMID) to explore information contents of deep neural models of music through rate reduction of latent representation streams, which is contrasted with hight rate information dynamics of the musical surface. This approach is partially motivated by rate-distortion theories of human cognition, providing a framework for exploring possible relations between imaginary anticipations existing in the listener's or composer's mind, and the information dynamics of the sensory (acoustic) or symbolic score data. In the paper the DMID framework is demonstrated using several experiments with symbolic (MIDI) and acoustic (spectral) music representations. We use variational encoding to learn a latent representation of the musical surface. This embedding is further reduced using a bit-allocation method into a second stream of low bit-rate encoding. The combined loss includes temporal information in terms of predictive properties for each encoding stream, and accuracy loss measured in terms of mutual information between the encoding at low rate and the high rate surface representations. For the case of counterpoint, we also study the mutual information between two voices in a musical piece at different levels of information reduction.The DMID framework allows to explore aspects of computational creativity in terms of juxtaposition of latent/imaginary surprisal aspects of deeper structure with music surprisal on the surface level, done in a manner that is quantifiable and computationally tractable. The relevant information theory modeling and analysis methods are discussed in the paper, suggesting that a trade off between compression and prediction play an important factor in the analysis and design of creative musical systems.

show abstract

“…The build procedure of XMusic will be introduced in Section 4.1.1. Inspired by contentbased music recommendation [3,17,18,20], we aim to leverage song content features, including audio, metadata, generated content, etc., to learn high-quality song representations supervised by the song-to-song interactions. However, we notice the imbalance of supervision signals between popular and less popular music, also known as the long tail effect.…”

Section: Introductionmentioning

confidence: 99%

Bootstrapping Contrastive Learning Enhanced Music Cold-Start Matching

Zhao

Zhang

Xiao

et al. 2023

Companion Proceedings of the ACM Web Conference 2023

View full text Add to dashboard Cite

We study a particular matching task we call Music Cold-Start Matching. In short, given a cold-start song request, we expect to retrieve songs with similar audiences and then fastly push the cold-start song to the audiences of the retrieved songs to warm up it. However, there are hardly any studies done on this task. Therefore, in this paper, we will formalize the problem of Music Cold-Start Matching detailedly and give a scheme. During the offline training, we attempt to learn high-quality song representations based on song content features. But, we find supervision signals typically follow power-law distribution causing skewed representation learning. To address this issue, we propose a novel contrastive learning paradigm named Bootstrapping Contrastive Learning (BCL) to enhance the quality of learned representations by exerting contrastive regularization. During the online serving, to locate the target audiences more accurately, we propose Clustering-based Audience Targeting (CAT) that clusters audience representations to acquire a few cluster centroids and then locate the target audiences by measuring the relevance between the audience representations and the cluster centroids. Extensive experiments on the offline dataset and online system demonstrate the effectiveness and efficiency of our method. Currently, we have deployed it on NetEase Cloud Music, affecting millions of users. CCS CONCEPTS• Information systems → Recommender systems.

show abstract

Learning Audio Embeddings with User Listening Data for Content-Based Music Recommendation

Cited by 28 publications

References 10 publications

Zero-Shot Audio Source Separation through Query-Based Learning from Weakly-Labeled Data

Zero-Shot Audio Source Separation through Query-Based Learning from Weakly-Labeled Data

Deep Music Information Dynamics Novel Framework for Reduced Neural-Network Music Representation with Applications to Midi and Audio Analysis and Improvisation

Bootstrapping Contrastive Learning Enhanced Music Cold-Start Matching

Contact Info

Product

Resources

About