A spark-based parallel distributed posterior decoding algorithm for big data hidden Markov models decoding problem

Sassi, Imad; Anter, Samir; Bekkhoucha, Abdelkrim

doi:10.11591/ijai.v10.i3.pp789-800

Cited by 5 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DAN is a type of deep learning, which is advanced machine learning. Deep learning is widely employed in artificial intelligence domains such as medicine [18], images processing [19], information and sound application [20], computer vision [21], and many more applications [15], [22]- [28]. In this paper, the DAN is suggested, implemented and evaluated.…”

Section: Deep Autoencoder Networkmentioning

confidence: 99%

“…From results, the suggested method was active in additive white gaussian noise (AWGN) channels, frequency-selective channels and rayleigh fading channels [14]. Sassi et al [15] modified posterior decoding algorithm to solve hidden Markov models by machine learning algorithms which deal with big data tasks. The authors succeeded in improving an algorithm to reduce time complexity and yield effective outcomes in  ISSN: 2302-9285 speedup, running time and high data volume parallelization efficiency.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Signal multiple encodings by using autoencoder deep learning

Anaz

Al-Ridha²,

Al-Nima³

2023

Bulletin EEI

View full text Add to dashboard Cite

Encryption is a substantial phase in information security. It permits only approved persons to get private information. This study suggests a signal multi-encryptions system (SMES) technique for coding and decoding signals created by a deep autoencoder network (DAN). The DAN of four layers is employed for a coding package of signals multiple times before decoding or restructuring the original signals again. The suggested SMES offers a high level of security as it can produce and exploit multiple encryptions for signals. Many statistical calculations are applied to measure the reliability of the system. The outcomes are promising where noteworthy encryptions-decryptions are obtained.

show abstract

Section: Deep Autoencoder Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Signal multiple encodings by using autoencoder deep learning

Anaz

Al-Ridha²,

Al-Nima³

2023

Bulletin EEI

View full text Add to dashboard Cite

show abstract

“…It is 100 times faster than hadoop. Spark provides a combination of fault-tolerance, in-memory processing, scalability, and speed [14], [15]. The cluster is managed by a cluster manager like yet another resource negotiator (YARN), Mesos, or spark's standalone cluster manager.…”

Section: Sparkmentioning

confidence: 99%

Architecting a machine learning pipeline for online traffic classification in software defined networking using spark

Samaan

Jeiad

2023

IJ-AI

View full text Add to dashboard Cite

Precise traffic classification is essential to numerous network functionalities such as routing, network management, and resource allocation. Traditional classification techniques became insufficient due to the massive growth of network traffic that requires high computational costs. The arising model of software defined networking (SDN) has adjusted the network architecture to get a centralized controller that preserves a global view over the entire network. This paper proposes a model for SDN traffic classification based on machine learning (ML) using the Spark framework. The proposed model consists of two phases; learning and deployment. A ML pipeline is constructed in the learning phase, consisting of a set of stages combined as a single entity. Three ML models are built and evaluated; decision tree, random forest, and logistic regression, for classifying a well-known 75 applications, including Google and YouTube, accurately and in a short time scale. A dataset consisting of 3,577,296 flows with 87 features is used for training and testing the models. The decision tree model is elected for deployment according to the performance results, which indicate that it has the best accuracy with 0.98. The performance of the proposed model is compared with the state-of-the-art works, and better accuracy result is reported.

show abstract

“…Similarly, Spark can store/write the data in all the mentioned data storages which means it has a diversity of data reading and writing from various data sources [25]. Also, the data structure of Apache Spark fundamentally consists of three types of data which are [18], [26], [27]: a. Resilient distributed datasets (RDD): spark uses a particular data structure known as RDD which is a logical collection of data and separated over machines. RDD is Spark's primary abstraction, which is a faulttolerant collection of elements that can be worked in parallel b.…”

Section: Spark Data Access and Data Structurementioning

confidence: 99%

Distributed big data analysis using spark parallel data processing

Omar

Jumaa

2022

Bulletin EEI

View full text Add to dashboard Cite

Nowadays, the big data marketplace is rising rapidly. The big challenge is finding a system that can store and handle a huge size of data and then processing that huge data for mining the hidden knowledge. This paper proposed a comprehensive system that is used for improving big data analysis performance. It contains a fast big data processing engine using Apache Spark and a big data storage environment using Apache Hadoop. The system tests about 11 Gigabytes of text data which are collected from multiple sources for sentiment analysis. Three different machine learning (ML) algorithms are used in this system which is already supported by the Spark ML package. The system programs were written in Java and Scala programming languages and the constructed model consists of the classification algorithms as well as the pre-processing steps in a figure of ML pipeline. The proposed system was implemented in both central and distributed data processing. Moreover, some datasets manipulation manners have been applied in the system tests to check which manner provides the best accuracy and time performance. The results showed that the system works efficiently for treating big data, it gains excellent accuracy with fast execution time especially in the distributed data nodes.

show abstract

A spark-based parallel distributed posterior decoding algorithm for big data hidden Markov models decoding problem

Cited by 5 publications

References 31 publications

Signal multiple encodings by using autoencoder deep learning

Signal multiple encodings by using autoencoder deep learning

Architecting a machine learning pipeline for online traffic classification in software defined networking using spark

Distributed big data analysis using spark parallel data processing

Contact Info

Product

Resources

About