Towards a Scalable kNN CF Algorithm: Exploring Effective Applications of Clustering

Rashid, Al Mamunur; Lam, Shyong K.; LaPitz, Adam; Karypis, George; Riedl, John

doi:10.1007/978-3-540-77485-3_9

Cited by 13 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…x = (weekday?, speed, flow, occupancy, visibility) , (5) and the target value y is the speed data in the next moment. The feature weekday?…”

Section: A Gaussian Process Regressionmentioning

confidence: 99%

“…Large-scale Learning via kNN and Gaussian Process k-nearest neighbors is a simple non-parametric algorithm which can solve both classification and regression problems. Apart from classification and regression, searching for nearest neighbors is the key procedure of many useful algorithms such as recommendation [5], dimensionality reduction [6], computer networking [7], and so on. In the most naïve implementation, kNN, as a lazy approach needs O(nD) to search for closest neighbors given a new data for prediction where n is the number of data points and D is the dimensionality.…”

Section: Past Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient traffic speed forecasting based on massive heterogenous historical data

Chen

Pao

Lee

2014

2014 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Drivers dream of foreseeing traffic condition to enjoy efficient driving experience at all times. Given the historical patterns for different locations and different time, people should be able to guess the possible traffic speed in a near future moment. What is difficult and interesting for this task is that we need to filter the useful data that could help us for the next moment traffic speed prediction from a massive amount of historical data. On the other hand, the traffic condition could be highly dynamic and we can only give a reliable traffic prediction by using the most updated model for prediction. This implies that frequent retraining is necessary. To conquer the task, we propose a lazy learning approach for traffic speed prediction given massive historical data. The approach integrates the kNN and Gaussian process regression for efficient and robust traffic speed prediction. kNN can help us to select the most informative data for Gaussian process Regression using a big data framework. Thanks for the most recent progress of big data research, the processing of massive data for prediction in close to real time has become possible now compared to any time in the past. We aim at using a Hadoop framework for the prediction given heterogeneous data including traffic data such as speed, flow, occupancy, and weather data.

show abstract

“…x = (weekday?, speed, flow, occupancy, visibility) , (5) and the target value y is the speed data in the next moment. The feature weekday?…”

Section: A Gaussian Process Regressionmentioning

confidence: 99%

Section: Past Workmentioning

confidence: 99%

Efficient traffic speed forecasting based on massive heterogenous historical data

Chen

Pao

Lee

2014

2014 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

show abstract

“…The basic user-based collaborative filtering (uCF) formula as described in [2] [4] [5] uses PCC to predict the rating of user, u, on item, m, as follows:…”

Section: Related Workmentioning

confidence: 99%

A new adaptive framework for collaborative filtering prediction

Almosallam

Shang

2008

2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence)

View full text Add to dashboard Cite

Collaborative filtering is one of the most successful techniques for recommendation systems and has been used in many commercial services provided by major companies including Amazon, TiVo and Netflix. In this paper we focus on memory-based collaborative filtering (CF). Existing CF techniques work well on dense data but poorly on sparse data. To address this weakness, we propose to use z-scores instead of explicit ratings and introduce a mechanism that adaptively combines global statistics with item-based values based on data density level. We present a new adaptive framework that encapsulates various CF algorithms and the relationships among them. An adaptive CF predictor is developed that can self adapt from user-based to item-based to hybrid methods based on the amount of available ratings. Our experimental results show that the new predictor consistently obtained more accurate predictions than existing CF methods, with the most significant improvement on sparse data sets. When applied to the Netflix Challenge data set, our method performed better than existing CF and singular value decomposition (SVD) methods and achieved 4.67% improvement over Netflix's system.

show abstract

“…Ideally, partitioning will improve the quality of collaborative filtering predictions and increase the scalability of collaborative filtering systems. For example, Rashid et al [13] present a CLUSTKNN that is well suitable for large datasets. They use a clustering model to compress the data first, and then use a simple nearest neighbor-based approach to generate recommendation.…”

Section: Introductionmentioning

confidence: 98%

Item-Based Clustering Collaborative Filtering Algorithm under High-Dimensional Sparse Data

Yao

Zhang

2009

2009 International Joint Conference on Computational Sciences and Optimization

View full text Add to dashboard Cite

This paper proposes a novel algorithm named itembased clustering recommendation algorithm (IBCRA) for reducing the poor recommendation quality due to the data sparsity and high dimension. Specifically, on the basis of high-dimensions data clustering algorithms, the IBCRA uses the rating data sparse difference and item categories in the rating dataset to construct a measuring formula for calculating dataset difference, where the formula is used for item clustering in user-item rating array. Then the IBCRA calculates item similarity and searches for k-nearest neighbors of target item based on the outcome of item clustering. Finally it predicts the ratings for those no rating items in dataset and so generates recommendations. The experimental results show, in perspective of the accuracy and speed of convergence, the IBCRA has improved the recommendation quality in collaborative filtering recommendation. Therefore, it can be used to recommend the products in ecommerce recommending systems.

show abstract

Towards a Scalable kNN CF Algorithm: Exploring Effective Applications of Clustering

Cited by 13 publications

References 16 publications

Efficient traffic speed forecasting based on massive heterogenous historical data

Efficient traffic speed forecasting based on massive heterogenous historical data

A new adaptive framework for collaborative filtering prediction

Item-Based Clustering Collaborative Filtering Algorithm under High-Dimensional Sparse Data

Contact Info

Product

Resources

About