A Hybrid Distributed Collaborative Filtering Recommender Engine Using Apache Spark

Panigrahi, Sasmita; Lenka, Rakesh Kumar; Stitipragyan, Ananya

doi:10.1016/j.procs.2016.04.214

Cited by 58 publications

(23 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In another related work, Panigrahi et al [6] proposed a hybrid solution to implement recommender a system using collaborative filtering and clustering techniques like K-means. It is based on in-memory computation of Apache Spark as big data platform allowed speed up the running time to make recommendation.…”

Section: Related Workmentioning

confidence: 99%

Large-scale e-learning recommender system based on Spark and Hadoop

et al. 2019

View full text Add to dashboard Cite

The computing environment for human learning is changing rapidly, due to the emergence of new information and communication technology such as big data [1] and cloud computing [2]. Furthermore, the learning methods are changing every day. Therefore, e-learning systems need to develop more techniques and tools to meet the increased needs of millions of learners around the world. This article exposes a smart recommender system applied in an online learning environment in order to be able to provide personalized courses and guide students to select more suitable courses. For example, emailing or sending notifications through the user interface of distance learning platform, to students who follow courses in a specific field

show abstract

Section: Related Workmentioning

confidence: 99%

Large-scale e-learning recommender system based on Spark and Hadoop

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Implicitly, the metadata is stored in Apache Derby, but other relational databases (e.g., MySQL) can be used. Hive can use Spark [70], as well as MapReduce or Tez, as its execution engine [71]. …”

Section: System Architecture and Componentsmentioning

confidence: 99%

Hadoop Oriented Smart Cities Architecture

Dıaconıța

Bologa

2018

Sensors

View full text Add to dashboard Cite

A smart city implies a consistent use of technology for the benefit of the community. As the city develops over time, components and subsystems such as smart grids, smart water management, smart traffic and transportation systems, smart waste management systems, smart security systems, or e-governance are added. These components ingest and generate a multitude of structured, semi-structured or unstructured data that may be processed using a variety of algorithms in batches, micro batches or in real-time. The ICT architecture must be able to handle the increased storage and processing needs. When vertical scaling is no longer a viable solution, Hadoop can offer efficient linear horizontal scaling, solving storage, processing, and data analyses problems in many ways. This enables architects and developers to choose a stack according to their needs and skill-levels. In this paper, we propose a Hadoop-based architectural stack that can provide the ICT backbone for efficiently managing a smart city. On the one hand, Hadoop, together with Spark and the plethora of NoSQL databases and accompanying Apache projects, is a mature ecosystem. This is one of the reasons why it is an attractive option for a Smart City architecture. On the other hand, it is also very dynamic; things can change very quickly, and many new frameworks, products and options continue to emerge as others decline. To construct an optimized, modern architecture, we discuss and compare various products and engines based on a process that takes into consideration how the products perform and scale, as well as the reusability of the code, innovations, features, and support and interest in online communities.

show abstract

“…Fig. 2 Source processing architecture Spark is a superior performance of the distributed computing framework.RDD is the distributed memory data abstraction, also known as flexible distributed data sets, and a lot of machine data were distinguished, RDD comprised of multiple partition, according to the number of which is cut into pieces, then each partition can correspond to a task, eventually reaching the effect of parallel computing [10].…”

Section: Experiments and Evaluationmentioning

confidence: 99%

Hybrid recommendation and parallelization of movies based on spark

Zhou¹,

Liu²

2018

Proceedings of the 2017 4th International Conference on Machinery, Materials and Computer (MACMC 2017)

View full text Add to dashboard Cite

Abstract:With the exponential growth of Internet data,the traditional stand-alone computational model has been unable to solve the real-time precise recommendation items in a complex and huge data,and the defect of traditional recommendation algorithm has become more obvious,this paper studies the collaborative filtering algorithm and matrix decomposition method,designs a parallel computing architecture based on spark,and a movie recommendation based on hybrid recommendation algorithm [1],the experimental results show that in a certain extent improves the recommendation accuracy and scalability,and has good acceleration effect.

show abstract

A Hybrid Distributed Collaborative Filtering Recommender Engine Using Apache Spark

Cited by 58 publications

References 6 publications

Large-scale e-learning recommender system based on Spark and Hadoop

Large-scale e-learning recommender system based on Spark and Hadoop

Hadoop Oriented Smart Cities Architecture

Hybrid recommendation and parallelization of movies based on spark

Contact Info

Product

Resources

About