Apache Spark a Big Data Analytics Platform for Smart Grid

Shyam, Radhey; Ganesh, H B Bharathi; Kumar, S Sachin; Poornachandran, Prabaharan; Soman, K P

doi:10.1016/j.protcy.2015.10.085

Cited by 124 publications

(51 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…(1) OR-Library benchmarks: this class includes 65 small and medium size randomly generated problems that were frequently used in the literature. They are available in the OR-Library and are described in Table 1.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Binary Cuckoo Search Big Data Algorithm Applied to Large‐Scale Crew Scheduling Problems

et al. 2018

View full text Add to dashboard Cite

The progress of metaheuristic techniques, big data, and the Internet of things generates opportunities to performance improvements in complex industrial systems. This article explores the application of Big Data techniques in the implementation of metaheuristic algorithms with the purpose of applying it to decision-making in industrial processes. This exploration intends to evaluate the quality of the results and convergence times of the algorithm under different conditions in the number of solutions and the processing capacity. Under what conditions can we obtain acceptable results in an adequate number of iterations? In this article, we propose a cuckoo search binary algorithm using the MapReduce programming paradigm implemented in the Apache Spark tool. The algorithm is applied to different instances of the crew scheduling problem. The experiments show that the conditions for obtaining suitable results and iterations are specific to each problem and are not always satisfactory.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Spark has a high performance in parallel computing, being used in machine learning algorithms [65], imaging processing [66], bioinformatics [67], computational intelligence [9], astronomy [68], medical information [69], and so on.…”

Section: Spark Distributed Frameworkmentioning

confidence: 99%

A Binary Cuckoo Search Big Data Algorithm Applied to Large‐Scale Crew Scheduling Problems

et al. 2018

View full text Add to dashboard Cite

show abstract

“…This has been used in machine learning algorithms [47], imaging processing [48], bioinformatics [49], computational intelligence [50], astronomy [51], medical information [52], and so on.…”

Section: Spark Distributed Frameworkmentioning

confidence: 99%

A Distributed K-Means Segmentation Algorithm Applied to Lobesia botrana Recognition

2017

View full text Add to dashboard Cite

Early detection of Lobesia botrana is a primary issue for a proper control of this insect considered as the major pest in grapevine. In this article, we propose a novel method for L. botrana recognition using image data mining based on clustering segmentation with descriptors which consider gray scale values and gradient in each segment. This system allows a 95 percent of L. botrana recognition in non-fully controlled lighting, zoom, and orientation environments. Our image capture application is currently implemented in a mobile application and subsequent segmentation processing is done in the cloud.

show abstract

“…San et al [11] conducted a study in the field of smart grid research. They proposed a complete automation system, where large pool of sensors is embedded in the existing power grids system for controlling and monitoring it by utilizing modern information technologies.…”

Section: ) Streaming Processingmentioning

confidence: 99%

Real-Time Analysis of Students’ Activities on an E-Learning Platform based on Apache Spark

Chaffai¹,

Hassouni²,

Anoun³

2017

ijacsa

View full text Add to dashboard Cite

Abstract-Real time analytics is the capacity to extract valuables insights from data that comes continuously from activities on the web or network sensors. It is largely used in web based business to drive decisions based on user's experiences, such dynamic pricing and personalized advertising. Many universities have adopted web based learning in their learning process. They use data-mining techniques to better understand students' behavior, and most of the tools developed are based on historical and stored data, and do not allow real time reactivity. Online activities of learners generate at high speed a huge amount of data in form of users' interactions which have all characteristics to be considered as Big data. Deal with volume and velocity of these data in order to inform and enable decisions-makers to act at right time lead us to use new methods to capture E-Learning data, and process it in real time.This paper focuses on the design and implementation of modern and hybrid real time data pipeline architecture using Apache Flume to collect data, Apache Spark as an unified engine computation for performing analytics on students' activities data and Apache Hive as a data warehouse for storing the processed data and for use by various reporting tools. To conceive this platform we conduct an experiment on Moodle database source.

show abstract

Apache Spark a Big Data Analytics Platform for Smart Grid

Cited by 124 publications

References 12 publications

A Binary Cuckoo Search Big Data Algorithm Applied to Large‐Scale Crew Scheduling Problems

A Binary Cuckoo Search Big Data Algorithm Applied to Large‐Scale Crew Scheduling Problems

A Distributed K-Means Segmentation Algorithm Applied to Lobesia botrana Recognition

Real-Time Analysis of Students’ Activities on an E-Learning Platform based on Apache Spark

Contact Info

Product

Resources

About