A Comprehensive Review on Big Data Challenges

Bharany, Salil; Taleb, Nasser; Sadiq, Muhammad Tariq; Kanwal, Nayab; Abdelhakim, Mohamed; Ghazal, Taher M.; Pradhan, Manas Ranjan; Rehman, Ateeq Ur

doi:10.1109/icbats57792.2023.10111375

Cited by 3 publications

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data Velocity : In order to derive useful insights from data streams in real-time as they are being created, high-performance computing skills are required (Bharany, 2023 ; Olawoyin et al, 2023 ).…”

Section: Introductionmentioning

confidence: 99%

Optimizing multi-objective task scheduling in fog computing with GA-PSO algorithm for big data application

Saad,

Enam,

Qureshi

2024

Front. Big Data

View full text Add to dashboard Cite

As the volume and velocity of Big Data continue to grow, traditional cloud computing approaches struggle to meet the demands of real-time processing and low latency. Fog computing, with its distributed network of edge devices, emerges as a compelling solution. However, efficient task scheduling in fog computing remains a challenge due to its inherently multi-objective nature, balancing factors like execution time, response time, and resource utilization. This paper proposes a hybrid Genetic Algorithm (GA)-Particle Swarm Optimization (PSO) algorithm to optimize multi-objective task scheduling in fog computing environments. The hybrid approach combines the strengths of GA and PSO, achieving effective exploration and exploitation of the search space, leading to improved performance compared to traditional single-algorithm approaches. The proposed hybrid algorithm results improved the execution time by 85.68% when compared with GA algorithm, by 84% when compared with Hybrid PWOA and by 51.03% when compared with PSO algorithm as well as it improved the response time by 67.28% when compared with GA algorithm, by 54.24% when compared with Hybrid PWOA and by 75.40% when compared with PSO algorithm as well as it improved the completion time by 68.69% when compared with GA algorithm, by 98.91% when compared with Hybrid PWOA and by 75.90% when compared with PSO algorithm when various tasks inputs are given. The proposed hybrid algorithm results also improved the execution time by 84.87% when compared with GA algorithm, by 88.64% when compared with Hybrid PWOA and by 85.07% when compared with PSO algorithm it improved the response time by 65.92% when compared with GA algorithm, by 80.51% when compared with Hybrid PWOA and by 85.26% when compared with PSO algorithm as well as it improved the completion time by 67.60% when compared with GA algorithm, by 81.34% when compared with Hybrid PWOA and by 85.23% when compared with PSO algorithm when various fog nodes are given.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimizing multi-objective task scheduling in fog computing with GA-PSO algorithm for big data application

Saad,

Enam,

Qureshi

2024

Front. Big Data

View full text Add to dashboard Cite

show abstract

“…In the era of big data, traditional machine learning methods can be computationally burdensome and complex, making them unsuitable for processing such large-scale datasets. To achieve accurate predictions for data, traditional machine learning techniques often struggle to handle the challenges posed by big data, including its sheer volume, complexity, and high-dimensional nature [1]. On the other hand, data-driven methods utilizing deep learning have attracted interest due to their capacity to perform statistical analysis and information extraction automatically and successfully on large-scale, multi-source, and high-dimensional data., thereby overcoming the limitations of traditional prediction methods [2].…”

Section: Introductionmentioning

confidence: 99%

Enhanced transformer long short-term memory framework for datastream prediction

Dief,

Salem,

Rabie

et al. 2024

IJECE

View full text Add to dashboard Cite

In machine learning, datastream prediction is a challenging issue, particularly when dealing with enormous amounts of continuous data. The dynamic nature of data makes it difficult for traditional models to handle and sustain real-time prediction accuracy. This research uses a multi-processor long short-term memory (MPLSTM) architecture to present a unique framework for datastream regression. By employing several central processing units (CPUs) to divide the datastream into multiple parallel chunks, the MPLSTM framework illustrates the intrinsic parallelism of long short-term memory (LSTM) networks. The MPLSTM framework ensures accurate predictions by skillfully learning and adapting to changing data distributions. Extensive experimental assessments on real-world datasets have demonstrated the clear superiority of the MPLSTM architecture over previous methods. This study uses the transformer, the most recent deep learning breakthrough technology, to demonstrate how well it can handle challenging tasks and emphasizes its critical role as a cutting-edge approach to raising the bar for machine learning.

show abstract

A Critical Investigation into the Blockchain Technology and Its Present-Day Uses

Bharany,

Maashi

2023

2023 3rd International Conference on Computing and Information Technology (ICCIT)

View full text Add to dashboard Cite

A Comprehensive Review on Big Data Challenges

Cited by 3 publications

References 27 publications

Optimizing multi-objective task scheduling in fog computing with GA-PSO algorithm for big data application

Optimizing multi-objective task scheduling in fog computing with GA-PSO algorithm for big data application

Enhanced transformer long short-term memory framework for datastream prediction

A Critical Investigation into the Blockchain Technology and Its Present-Day Uses

Contact Info

Product

Resources

About