A novel approach for big data processing using message passing interface based on memory mapping

Dheyab, Saad Ahmed; Abdullah, Mohammed Najm; Abed, Buthainah F.

doi:10.1186/s40537-019-0275-3

Cited by 10 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“… Variety: occasionally, data come from various sources and in diverse forms, such as structured, unstructured, and semi-structured data [24]. As a result, it is vital to have systems that can handle various data models while preserving performance [25].  Veracity: it indicates the quality of data to be analyzed.…”

Section: B Big Datamentioning

confidence: 99%

A Review on Utilizing Big Data Techniques for Performance Improvement in Software Defined Networking

2023

IJCCCE

View full text Add to dashboard Cite

Traditional network abilities have a drastic shortage in the current networking world. Software-Defined Networking (SDN) is a revival development in the networking domain that provides separation of control and data planes, enlarges the data plane granularity, and simplifies the network devices. All these factors accelerate and automate the evolution of new services. However, when the SDN network topology becomes large, it poses new challenges in security, traffic management, and scalability due to the vast amounts of traffic data generated and the need for additional controllers to manage the significant number of networking devices. On the other hand, big data has become an attractive trend that can enhance network performance in general, specifically SDN. Both SDN and big data have gained great attraction from industry and academia. Traditionally, these two subjects have been studied separately in most of the preceding works. However, big data can impact the design and implementation of SDN thoroughly. This paper presents how big data can support SDN in various aspects, including intrusion detection, traffic monitoring, and controller scalability and resiliency. We suggest several approaches toward deeper cooperation between big data and SDN. Index Term – SDN, big data, Spark, graph database, traffic engineering, intrusion detection.

show abstract

Section: B Big Datamentioning

confidence: 99%

A Review on Utilizing Big Data Techniques for Performance Improvement in Software Defined Networking

2023

IJCCCE

View full text Add to dashboard Cite

show abstract

“…Intrusion detection techniques ensure the security of a network through monitoring, detecting, and responding to attacks. The major concern of an IDS is to identify both internal and external network threats [19] [12]. In a common way, an IDS can be said to consist of hardware components which require compatible software to be run [20] [13].…”

Section: Intrusion Detection Systemmentioning

confidence: 99%

“…From the computational perspective, the GHA [16,17] is beneficial because it can solve eigenvalue problems using iterative methods which requires no direct covariance matrix computation. This is more significant when there are many attributes in a given set [18,19]. Oja and Karhunen [16] demonstrated an incremental solution to finding the first eigenvector from data arriving in the form of serial data items presented as vectors [7,20].…”

Section: Generalized Hebbian Algorithm (Gha)mentioning

confidence: 99%

Fuzzy Generalized Hebbian Algorithm for Large-Scale Intrusion Detection System

Ali¹,

Aljanabi

Ahmed

2020

IJIE

View full text Add to dashboard Cite

The huge number of irrelevant and redundant data used in building intrusion detection systems (IDS) is one of the common issues in network intrusion detection systems. This paper proposed the use of Fuzzy Generalized Hebbian Algorithm as a novel data reduction method to overcome this problem of data redundancy in IDS. Two methods for dimensionality reduction (GHA and Fuzzy GHA) were used and compared in this study. This allowed retaining the most relevant traffic data information from the network. Furthermore, the K Nearest Neighbor algorithm was applied for the classification of the test connections into 2 categories (attack or normal). The investigations were carried out on the KDDCUP '99 dataset and the results showed the Fuzzy GHA method to perform better than GHA in the detection of both U2R and DoS attacks.

show abstract

“…This data is streaming everywhere through multiple devices, which are part of the Internet of Things (IoT). It includes sensors' data, devices that produce a massive amount of data [1], vehicles [2], phones and machines [3], social media, cyber-physical systems [4], business…”

Section: Introductionmentioning

confidence: 99%

LSDStrategy: A Lightweight Software-Driven Strategy for Addressing Big Data Variety of Multimedia Streaming

Khudhur

Jeiad

2022

IEEE Access

View full text Add to dashboard Cite

Many devices, users, and applications stream an irregular amount of varied data every second. This rapid generation of data continues at an enormous rate, constructing the big data that increase the need for solutions, despite resource constraints, to analyze and manipulate data. Current methods allocate cloud resources according to the characteristics of the data. Resource allocation requires a comprehensive view of the workload requirements. However, the data characteristics in big data streams are uncertain due to the random nature of data generation. Choosing and allocating the right resources to this stream is challenging. With the variety of big data streams, the stochastic nature of the stream led to unpredictable requirements and specifications. The critical issue is forecasting the workload to avoid the over-provisioning and under-provisioning of resources. Such forecasting needs an adequate dataset to describe the history logs of the incoming workload. A fast release for such a dataset provides a high chance of deploying forecasting at the right time. This paper addresses this issue with a novel strategy named LSDStrategy that analyzes the received multimedia stream based on its binary content using machine learning techniques with artificial and real datasets. LSDStrategy utilizes an evaluating voting technique to select the optimum classifier to trade off accuracy and prediction time as metrics. Multi-classifiers that have been built and tested include Decision Tree (DT), K-Nearest Neighbor (K-NN), and Random Forest (RF) over multi-content-based features. Experiments evaluated the performance of the adopted models and the selected features. According to experimental analysis, the DT approach provides a consistent performance for both artificial and realworld datasets for 85% and 81.3%, respectively. We deploy and evaluate the LSDStrategy efficiency on a regular specification server through a set of experiments using a synthetic stream. The experiments prove the LSDStrategy agility and adaptivity in identifying the multimedia-based workload type utilizing small chunks of load.

show abstract

A novel approach for big data processing using message passing interface based on memory mapping

Cited by 10 publications

References 37 publications

A Review on Utilizing Big Data Techniques for Performance Improvement in Software Defined Networking

A Review on Utilizing Big Data Techniques for Performance Improvement in Software Defined Networking

Fuzzy Generalized Hebbian Algorithm for Large-Scale Intrusion Detection System

LSDStrategy: A Lightweight Software-Driven Strategy for Addressing Big Data Variety of Multimedia Streaming

Contact Info

Product

Resources

About