RecoBERT: A Catalog Language Model for Text-Based Recommendations

Abstract

Language models that utilize extensive selfsupervised pre-training from unlabeled text, have recently shown to significantly advance the state-of-the-art performance in a variety of language understanding tasks. However, it is yet unclear if and how these recent models can be harnessed for conducting text-based recommendations. In this work, we introduce RecoBERT, a BERT-based approach for learning catalog-specialized language models for text-based item recommendations. We suggest novel training and inference procedures for scoring similarities between pairs of items, that don't require item similarity labels. Both the training and the inference techniques were designed to utilize the unlabeled structure of textual catalogs, and minimize the discrepancy between them. By incorporating four scores during inference, RecoBERT can infer text-based item-to-item similarities more accurately than other techniques. In addition, we introduce a new language understanding task for wine recommendations using similarities based on professional wine reviews. As an additional contribution, we publish annotated recommendations dataset crafted by human wine experts. Finally, we evaluate Re-coBERT and compare it to various state-of-theart NLP models on wine and fashion recommendations tasks.

Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference

“…Semantic similarity has been studied in many fields, such as computer vision (Parmar et al, 2018;Huang et al, 2017), recommender systems (Wang and Fu, 2020;Barkan et al, 2020aBarkan et al, , 2021Malkiel et al, 2020), and natural language processing (Devlin et al, 2019;Reimers and Gurevych, 2019;Mikolov et al, 2013). Recently, transformer-based Language Models (LMs) ushered significant performance gains in various natural language understanding tasks, but mainly on relatively short texts (Devlin et al, 2019;Liu et al, 2019).…”

Cold Start Revisited: A Deep Hybrid Recommender with Cold-Warm Item Harmonization

“…We employed two content analyzers: (1) For categorical features and tags, we used a single linear layer with a 100-dimensional output vector. (2) For textual description, we used a variant of the RecoBERT model [23] that produces an embedding of size 100. Finally, the multiview content analyzer produces a 40-dimensional output vector.…”

Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference

“…Semantic similarity has been studied in many fields, such as computer vision (Parmar et al, 2018;Huang et al, 2017), recommender systems (Wang and Fu, 2020;Barkan et al, 2020aBarkan et al, , 2021Malkiel et al, 2020), and natural language processing (Devlin et al, 2019;Reimers and Gurevych, 2019;Mikolov et al, 2013). Recently, transformer-based Language Models (LMs) ushered significant performance gains in various natural language understanding tasks, but mainly on relatively short texts (Devlin et al, 2019;Liu et al, 2019).…”