A new adaptive framework for collaborative filtering prediction

Abstract

Collaborative filtering is one of the most successful techniques for recommendation systems and has been used in many commercial services provided by major companies including Amazon, TiVo and Netflix. In this paper we focus on memory-based collaborative filtering (CF). Existing CF techniques work well on dense data but poorly on sparse data. To address this weakness, we propose to use z-scores instead of explicit ratings and introduce a mechanism that adaptively combines global statistics with item-based values based on data density level. We present a new adaptive framework that encapsulates various CF algorithms and the relationships among them. An adaptive CF predictor is developed that can self adapt from user-based to item-based to hybrid methods based on the amount of available ratings. Our experimental results show that the new predictor consistently obtained more accurate predictions than existing CF methods, with the most significant improvement on sparse data sets. When applied to the Netflix Challenge data set, our method performed better than existing CF and singular value decomposition (SVD) methods and achieved 4.67% improvement over Netflix's system.

A Survey of Recommender System Types and Its Classification

“…This works well on a sparse dataset.In [26] a cascading hybrid approach was proposed which combines the features, demographic information and ratings about an item and claimed to have addressed the shortcomings of both collaborative and content based filtering.A method that was proposed in [27] adds the concept of time context to its collaborative filtering algorithm. This enhancement has improved the performance and accuracy of the recommendations.Another method [28] effective on sparse data was proposed by Ibrahim et. al.…”