Improving Collaborative Metric Learning with Efficient Negative Sampling

Tran, Viet-Anh; Hennequin, Romain; Royo-Letelier, Jimena; Moussallam, Manuel

doi:10.1145/3331184.3331337

Cited by 29 publications

(29 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One limitation of CONVFIT, especially prominent in Full scenarios, is its quadratic time complexity. Future work will look into effective sampling strategies and adaptations towards more sampleefficient and quicker fine-tuning (Tran et al, 2019;Tian et al, 2020;O'Neill and Bollegala, 2021).…”

Section: Further Discussionmentioning

confidence: 99%

ConvFiT: Conversational Fine-Tuning of Pretrained Language Models

Vulić¹,

Su²,

Coope³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Transformer-based language models (LMs) pretrained on large text collections are proven to store a wealth of semantic knowledge. However, 1) they are not effective as sentence encoders when used off-the-shelf, and 2) thus typically lag behind conversationally pretrained (e.g., via response selection) encoders on conversational tasks such as intent detection (ID). In this work, we propose CON-VFIT, a simple and efficient two-stage procedure which turns any pretrained LM into a universal conversational encoder (after Stage 1 CONVFIT-ing) and task-specialised sentence encoder (after Stage 2). We demonstrate that 1) full-blown conversational pretraining is not required, and that LMs can be quickly transformed into effective conversational encoders with much smaller amounts of unannotated data; 2) pretrained LMs can be fine-tuned into task-specialised sentence encoders, optimised for the fine-grained semantics of a particular task. Consequently, such specialised sentence encoders allow for treating ID as a simple semantic similarity task based on interpretable nearest neighbours retrieval. We validate the robustness and versatility of the CON-VFIT framework with such similarity-based inference on the standard ID evaluation sets: CONVFIT-ed LMs achieve state-of-the-art ID performance across the board, with particular gains in the most challenging, few-shot setups.

show abstract

Section: Further Discussionmentioning

confidence: 99%

ConvFiT: Conversational Fine-Tuning of Pretrained Language Models

Vulić¹,

Su²,

Coope³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…We only keep the songs whose Essentia (and consequently AVD) features can be calculated, which corresponds to a total of 204, 316 songs [21]. The playcount data is binarized by retaining values of five or higher as implicit feedback [31]. As in [8,15], in order to keep the computational burden low, we retain the top songs and users (sorted by playcounts) and we remove inactive users and items (that is, we only keep users who listened to at least 20 songs, and songs which have been listened to by at least 50 users).…”

Section: Protocolmentioning

confidence: 99%

Leveraging the Structure of Musical Preference in Content-Aware Music Recommendation

Magron

Févotte

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

State-of-the-art music recommendation systems are based on collaborative filtering, which predicts a user's interest from his listening habits and similarities with other users' profiles. These approaches are agnostic to the song content, and therefore face the cold-start problem: they cannot recommend novel songs without listening history. To tackle this issue, content-aware recommendation incorporates information about the songs that can be used for recommending new items. Most methods falling in this category exploit either userannotated tags, acoustic features or deeply-learned features. Consequently, these content features do not have a clear musical meaning, thus they are not necessarily relevant from a musical preference perspective. In this work, we propose instead to leverage a model of musical preference which originates from the field of music psychology. From low-level acoustic features we extract three factors (arousal, valence and depth), which have been shown appropriate for describing musical taste. Then we integrate those into a collaborative filtering framework for content-aware music recommendation. Experiments conducted on large-scale data show that this approach is able to address the cold-start problem, while using a compact and meaningful set of musical features.

show abstract

“…Different from the RME which jointly decomposes the user-item rating matrix and the user-user co-occurrence matrix, we utilize the users' co-occurrence patterns to distinguish sets of users with extremely similar or dissimilar consumption behaviors and combine social information to change their relative positions in the metric space. It is worth mentioning that negative sampling is not usually used when calculating the similarity of users' consumption behavior, but several works [20,21] have studied the implications of negative sampling as well as various methods to improve the quality of recommendation. Contrary to previous studies [8,9] which only used the PMI formula to capture the positive similarity between users, we consider negative sampling of user similarity and find the list of users with extremely dissimilar consumption behavior for each user.…”

Section: Motivationmentioning

confidence: 99%

A Social Recommendation Based on Metric Learning and Users’ Co-Occurrence Pattern

2021

View full text Add to dashboard Cite

For personalized recommender systems, matrix factorization and its variants have become mainstream in collaborative filtering. However, the dot product in matrix factorization does not satisfy the triangle inequality and therefore fails to capture fine-grained information. Metric learning-based models have been shown to be better at capturing fine-grained information than matrix factorization. Nevertheless, most of these models only focus on rating data and social information, which are not sufficient for dealing with the challenges of data sparsity. In this paper, we propose a metric learning-based social recommendation model called SRMC. SRMC exploits users’ co-occurrence patterns to discover their potentially similar or dissimilar users with symmetric relationships and change their relative positions to achieve better recommendations. Experiments on three public datasets show that our model is more effective than the compared models.

show abstract

Improving Collaborative Metric Learning with Efficient Negative Sampling

Cited by 29 publications

References 19 publications

ConvFiT: Conversational Fine-Tuning of Pretrained Language Models

ConvFiT: Conversational Fine-Tuning of Pretrained Language Models

Leveraging the Structure of Musical Preference in Content-Aware Music Recommendation

A Social Recommendation Based on Metric Learning and Users’ Co-Occurrence Pattern

Contact Info

Product

Resources

About