Solving cold-start problem in large-scale recommendation engines: A deep learning approach

Yuan, Jianbo; Shalaby, Walid; Korayem, Mohammed; Chuan-En, Lin, David; AlJadda, Khalifeh; Luo, Jiebo

doi:10.1109/bigdata.2016.7840810

Cited by 39 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yuan et al [27] examine the real-world data problem of matching users to job postings, where items are time sensitive and new items are very frequent. They make the case that high performance techniques that require item labels can be generalised to cold-start items by pairing labelled and unlabelled items based on the similarity of their content.…”

Section: Sparsity and Cold-startsmentioning

confidence: 99%

“…The resulting dense matrix can be used to recommend items based on a given user's past interactions [15]. However, due to dependence on user interactions, collaborative approaches present issues when items are time sensitive or competitive as items may not remain valid long enough to accumulate a significant user record [27]. Further, this approach can result in positive feedback loops where items with more numerous or diverse interaction histories are more frequently shown to users; this virality effect can result in a few generic or broadly applicable documents being disproportionately recommended, while newer are not promoted due to less existent user behaviour data [27].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Recommendations from cold starts in big data

Ralph

Li²,

Wills

et al. 2020

Computing

View full text Add to dashboard Cite

This paper examines the challenging problem of new user cold starts in subset labelled and extremely sparsely labelled big data. We introduce a new Isle of Wight Supply Chain (IWSC) dataset demonstrating these characteristics. We also introduce a new technique addressing these challenges, the Transitive Semantic Relationships (TSR) model, which infers potential relationships from user and item text content and few labelled examples. We perform both implicit and explicit evaluation of TSR as a recommender system and from new user cold starts we achieve a hit-rate@10 of 77% on a collection of 630 items with only 376 supply-chain consumer labels, and 67% with only 142 supply-chain supplier labels, demonstrating a high level of performance even with extremely few labels in challenging cold-start scenarios. TSR is suitable for any dataset featuring few labels and user and item content, where similarity of content indicates similar relationship forming capability. TSR can be used as a standalone recommender system or to complement existing high-performance recommender models that require more labels or do not support cold starts.

show abstract

Section: Sparsity and Cold-startsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Recommendations from cold starts in big data

Ralph

Li²,

Wills

et al. 2020

Computing

View full text Add to dashboard Cite

show abstract

“…To date, collaborative filtering (CF) methods (Koren and Bell 2015) lie at the core of most real-word movie recommendation engines, due to their state-of-the-art accuracy (McFee et al 2012;Yuan et al 2016). In most video-streaming services, however, new movies and TV series are continuously added.…”

Section: New Item Cold-start Recommendation In the Movie Domainmentioning

confidence: 99%

Movie genome: alleviating new item cold start in movie recommendation

Deldjoo

Dacrema

Constantin

et al. 2019

User Model User-Adap Inter

View full text Add to dashboard Cite

As of today, most movie recommendation services base their recommendations on collaborative filtering (CF) and/or content-based filtering (CBF) models that use metadata (e.g., genre or cast). In most video-on-demand and streaming services, however, new movies and TV series are continuously added. CF models are unable to make predictions in such a scenario, since the newly added videos lack interactions-a problem technically known as new item cold start (CS). Currently, the most common approach to this problem is to switch to a purely CBF method, usually by exploiting textual metadata. This approach is known to have lower accuracy than CF because it ignores useful collaborative information and relies on human-generated textual metadata, which are expensive to collect and often prone to errors. User-generated content, such as tags, can also be rare or absent in CS situations. In this paper, we introduce a new movie recommender system that addresses the new item problem in the movie domain by (i) integrating state-of-the-art audio and visual descriptors, which can be automatically extracted from video content and constitute what we call the movie genome; (ii) exploiting an effective data fusion method named canonical correlation analysis, which was successfully tested in our previous works Deldjoo et alto better exploit complementary information between different modalities; (iii) proposing a two-step hybrid approach which trains a CF model on warm items (items with interactions) and leverages the learned model on the movie genome to recommend cold items (items without interactions). Experimental validation is carried out using a system-centric study on a large-scale, real-world movie recommendation dataset both in an absolute cold start and in a cold to warm transition; and a user-centric online experiment measuring different subjective aspects, such as satisfaction and diversity. Results show the benefits of this approach compared to existing approaches.

show abstract

“…We utilize content in two different scenarios by distinguishing between the 'cold-start' situation for jobs and users. In the case of a new job, we compute a content-based similarity measure between jobs by building a neural network that learns job embeddings [10]. This way, we guarantee an association between jobs even when they do not share any user interactions.…”

Section: Related Workmentioning

confidence: 99%

“…In order to reduce the sparsity in our recommendation graph, we densify the graph by creating an edge representing a content-based similarity score between pairs of jobs using their descriptions through a Deep Learning Matcher (DLM) [10]. DLM works by training a neural network to generate a distributed representation of each job (a.k.a.…”

Section: ) Job Descriptionsmentioning

confidence: 99%

Help me find a job: A graph-based approach for job recommendation at scale

Shalaby

AlAila

Korayem

et al. 2017

2017 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

Online job boards are one of the central components of modern recruitment industry. With millions of candidates browsing through job postings everyday, the need for accurate, effective, meaningful, and transparent job recommendations is apparent more than ever. While recommendation systems are successfully advancing in variety of online domains by creating social and commercial value, the job recommendation domain is less explored. Existing systems are mostly focused on content analysis of resumes and job descriptions, relying heavily on the accuracy and coverage of the semantic analysis and modeling of the content in which case, they end up usually suffering from rigidity and the lack of implicit semantic relations that are uncovered from users' behavior and could be captured by Collaborative Filtering (CF) methods. Few works which utilize CF do not address the scalability challenges of real-world systems and the problem of cold-start. In this paper, we propose a scalable item-based recommendation system for online job recommendations. Our approach overcomes the major challenges of sparsity and scalability by leveraging a directed graph of jobs connected by multi-edges representing various behavioral and contextual similarity signals. The short lived nature of the items (jobs) in the system and the rapid rate in which new users and jobs enter the system make the cold-start a serious problem hindering CF methods. We address this problem by harnessing the power of deep learning in addition to user behavior to serve hybrid recommendations. Our technique has been leveraged by CareerBuilder.com which is one of the largest job boards in the world to generate high-quality recommendations for millions of users.

show abstract

Solving cold-start problem in large-scale recommendation engines: A deep learning approach

Cited by 39 publications

References 25 publications

Recommendations from cold starts in big data

Recommendations from cold starts in big data

Movie genome: alleviating new item cold start in movie recommendation

Help me find a job: A graph-based approach for job recommendation at scale

Contact Info

Product

Resources

About