A Study on Big Data Processing Frameworks: Spark and Storm

Deshai, N.; Venkataramana, S.; Sekhar, B. V. D. S.; Srinivas, K.; Varma, Girish

doi:10.1007/978-981-32-9690-9_43

Cited by 5 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Machine learning is one of the very effective methods by which Big Data can be processed, visualized, and interpreted [17], [18]. In this study, for the execution of our model, we relied on the Spark framework since spark is capable of performing large processing tasks quickly and allows the distribution of tasks over several computers for processing [19].…”

Section: Methodsmentioning

confidence: 99%

Prediction of Quality of Water According to a Random Forest Classifier

Alomani¹,

Alhawiti²,

Alhakamy³

2022

IJACSA

View full text Add to dashboard Cite

Potable or drinking water is a daily life necessity for humans. The safety of this water is a concern in many regions around the world, since polluted waters are increasing and causing the spread of disease among populations. Continuous management and evaluation of the water which is meant for drinking is very essential and must be taken seriously. Often, the quality of water is evaluated through regular laboratory testing and analysis which can be tiresome and time consuming. On the other hand, advanced technologies using big data with the help of machine learning can have better results in terms of potability evaluation. For this reason, several studies have been conducted on predicting the quality of water and the several factors and classification that affect the prediction model. In this study, a random forest model was developed using PySpark classification to predict the potability of river water by relying on ten different features: pH, hardness, presence of solids, presence of chloramines, presence of sulfate, conductivity, organic carbon, trihalomethanes, turbidity, and finally potability. In addition, The developed model was able to predict water potability classification with a 1.0 accuracy, and 1.0 F1-score.

show abstract

Section: Methodsmentioning

confidence: 99%

Prediction of Quality of Water According to a Random Forest Classifier

Alomani¹,

Alhawiti²,

Alhakamy³

2022

IJACSA

View full text Add to dashboard Cite

show abstract

“…This study intends to address the issues and opportunities in the Agriculture sector by diving into the application of big data analytics, cloud computing, and parallel distributed processing [11][12][13][14][15][16]. It focuses on the critical role of these technologies in precision farming, crop monitoring, yield prediction, and forecasting, as well as the impact of frameworks like Hadoop and Spark in processing and analysing agricultural data for informed decision-making and optimised farming operations [17][18][19][20][21][22].…”

Section: Introductionmentioning

confidence: 99%

Enhancing Agricultural Operations: Big Data Analytics Using Distributed and Parallel Computing

Fatima,

Syeda Faiza Nasim,

Saad Ahmed

2023

IJEET

View full text Add to dashboard Cite

This comprehensive research investigates using distributed and parallel computing for big data analytics in agriculture to improve farming operations' sustainability, efficiency, and innovation. The paper emphasizes how big data analytics, cloud computing, and parallel distributed processing can revolutionize the agricultural industry. The research objectives include investigating the benefits and limitations of big data analytics in precision farming and crop monitoring, identifying the constraints of integrating big data analytics in agriculture and investigating the role of frameworks such as Hadoop and Spark in processing and analyzing agricultural data for informed decision-making and optimized farming operations. The methodology used in the paper is a literature review, which draws on various sources to provide insights into the topic matter. The findings indicate that big data analytics can considerably improve precision farming and crop monitoring; nevertheless, there are hurdles to incorporating big data analytics in agriculture, such as data privacy and security concerns. According to the study, frameworks such as Hadoop and Spark are crucial in processing and analyzing agricultural data for informed decision-making and better farming operations. Overall, this study offers useful insights into the possibilities of big data analytics and distributed and parallel computing in revolutionizing the agriculture industry.

show abstract