Reducing Data Complexity in Feature Extraction and Feature Selection for Big Data Security Analytics

Sisiaridis, Dimitrios; Markowitch, Olivier

doi:10.1109/icdis.2018.00014

Cited by 13 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, most of these items are constrained by a limited storage capacity and a short system. Managing big data therefore becomes a difficult IoT problem 8 …”

Section: Overviewmentioning

confidence: 99%

“…Managing big data therefore becomes a difficult IoT problem. 8 Cloud computing, on the other hand, is unanimously accepted firmly by the researchers and industrial sectors. Cloud computing requires the use of the enormous storage capacity and the processing of servers somewhere on the planet via internet connection.…”

Section: Contributionmentioning

confidence: 99%

“…Which is generally referred to the Cloud of Things (CoT) integration 4 . Within this framework, we need techniques that would enable artificial intelligence (AI) to reduce data dimensions and complexity 7–10 . These techniques can be executed in fog level since it is close to IoT devices.…”

Section: Introductionmentioning

confidence: 99%

“…4 Within this framework, we need techniques that would enable artificial intelligence (AI) to reduce data dimensions and complexity. [7][8][9][10] These techniques can be executed in fog level since it is close to IoT devices. Two commonly used methods for selecting features and removing features were implemented to increase machine learning (ML) efficiency and to decrease data dimensions.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A fog computing data reduce level to enhance the cloud of things performance

Moulahi

Khediri

Khan

et al. 2021

Int J Communication

View full text Add to dashboard Cite

Summary Devices in the Internet of Things (IoT) generate and gather a vast amount of data. Smart devices with constrained resources cannot execute large data‐based machine learning (ML) algorithms. Therefore, in this paper, we propose, analyze, and evaluate data reduction at the Fog level. As a matter of fact, the focus of this paper is twofold: The first objective is to process the reduced datasets by ML, and the second aim is to segregate the irrelevant data and preserve the quality of the ML models. The naïve Bayesian classifier is used for model analysis. For data (attributes) reduction, the state‐of‐the‐art approaches of CF Subset Evaluation, Info Gain Evaluation, Gain Ratio Attribute Evaluation, and the principal component analysis (PCA) approaches are employed. From the implementation point of view, the naïve Bayesian classifier is used to learn the class distribution and the correlation of the classes with the rest of the features. The naïve Bayesian model is generated using the full dataset without feature reduction. Feature selection algorithms then reduce features by 50%. This reduced set is also used to generate the naïve Bayesian model, which gives us a benchmark for comparing the results with the original unreduced dataset. From the results, we find that the performance of the reduced set is either increased or is almost similar to the full set of features. With this comprehensive experimental evaluation, we believe that data reduction can provide a blueprint for avoiding unnecessary data storage and processing. The performance of the ML models is not drastically affected by feature reduction and hence demonstrates the efficacy of the reduced ML models.

show abstract

“…In fact, most of these items are constrained by a limited storage capacity and a short system. Managing big data therefore becomes a difficult IoT problem 8 …”

Section: Overviewmentioning

confidence: 99%

Section: Contributionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A fog computing data reduce level to enhance the cloud of things performance

Moulahi

Khediri

Khan

et al. 2021

Int J Communication

View full text Add to dashboard Cite

show abstract

“…Typically, a big data platform is used to collect data on security events from IPS and maintain security logs over long-term periods. The big data platform can also be specialized in analyzing data and quickly recognizing cyber threats [35], [36]. This is because historical data collected over long-term periods in the platform can help investigate and respond to cyber threats.…”

Section: Big Data Platformmentioning

confidence: 99%

Cyber Threat Detection Based on Artificial Neural Networks Using Event Profiles

et al. 2019

View full text Add to dashboard Cite

One of the major challenges in cybersecurity is the provision of an automated and effective cyber-threats detection technique. In this paper, we present an AI technique for cyber-threats detection, based on artificial neural networks. The proposed technique converts multitude of collected security events to individual event profiles and use a deep learning-based detection method for enhanced cyber-threat detection. For this work, we developed an AI-SIEM system based on a combination of event profiling for data preprocessing and different artificial neural network methods, including FCNN, CNN, and LSTM. The system focuses on discriminating between true positive and false positive alerts, thus helping security analysts to rapidly respond to cyber threats. All experiments in this study are performed by authors using two benchmark datasets (NSLKDD and CICIDS2017) and two datasets collected in the real world. To evaluate the performance comparison with existing methods, we conducted experiments using the five conventional machine-learning methods (SVM, k-NN, RF, NB, and DT). Consequently, the experimental results of this study ensure that our proposed methods are capable of being employed as learning-based models for network intrusion-detection, and show that although it is employed in the real world, the performance outperforms the conventional machine-learning methods.

show abstract