Evolving meta-ensemble of classifiers for handling incomplete and unbalanced datasets in the cyber security domain

Folino, Gianluigi; Pisani, Francesco Sergio

doi:10.1016/j.asoc.2016.05.044

Cited by 27 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, in large datasets plagued by MCAR missing data, samples with missing values can be discarded without biasing the distribution of the remaining data. This study simulated missing rates of 10%, 20%, 30%, 40%, and 50% [9,15,20,32,35,41] to compare the proposed method to the imputation methods listed in the UCI datasets experiment. As shown in Eq.…”

Section: Data Pre-processingmentioning

confidence: 99%

“…The mean and mode are common statistical MVI technique measurements that typically require a short time to compute. However, machine learning MVI techniques, such as support vector machine (SVM), and random forest (RF) methods, require a long computation time to achieve high accuracy [20][21][22][23]. On the other hand, the k-nearest neighbor (KNN) technique [24] requires much less imputation time than other machine learning techniques [25][26][27][28].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Adaptive multiple imputations of missing values using the class center

et al. 2022

View full text Add to dashboard Cite

Big data has become a core technology to provide innovative solutions in many fields. However, the collected dataset for data analysis in various domains will contain missing values. Missing value imputation is the primary method for resolving problems involving incomplete datasets. Missing attribute values are replaced with values from a selected set of observed data using statistical or machine learning methods. Although machine learning techniques can generate reasonably accurate imputation results, they typically require longer imputation durations than statistical techniques. This study proposes the adaptive multiple imputations of missing values using the class center (AMICC) approach to produce effective imputation results efficiently. AMICC is based on the class center and defines a threshold from the weighted distances between the center and other observed data for the imputation step. Additionally, the distance can be an adaptive nearest neighborhood or the center to estimate the missing values. The experimental results are based on numerical, categorical, and mixed datasets from the University of California Irvine (UCI) Machine Learning Repository with introduced missing values rate from 10 to 50% in 27 datasets. The proposed AMICC approach outperforms the other missing value imputation methods with higher average accuracy at 81.48% which is higher than those of other methods about 9 – 14%. Furthermore, execution time is different from the Mean/Mode method, about seven seconds; moreover, it requires significantly less time for imputation than some machine learning approaches about 10 – 14 s.

show abstract

Section: Data Pre-processingmentioning

confidence: 99%

mentioning

confidence: 99%

Adaptive multiple imputations of missing values using the class center

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Considering all these issues, we choose to adopt CAGE-MetaCombiner, which employs a meta-ensemble model to operate efficiently with missing data, described in detail in [9], in our scalable architecture proposed in this paper. To synthesise, an ensemble is built for each different source of data (dataset) by using a distributed GP tool, used to generate the combiner function, i.e., CellulAr GEnetic programming (CAGE) [5], a distributed/parallel genetic programming (GP) implementation.…”

Section: Missing Data and Ensemble Of Classifiersmentioning

confidence: 99%

A Scalable Cybersecurity Framework for Anomaly Detection in User Behaviour

Folino¹,

Godano²,

Pisani³

2022

Preprint

View full text Add to dashboard Cite

Nowadays, the speed of the user and application logs is so quick that it is almost impossible to analyse them in real-time without using scalable systems and platforms. In cybersecurity, human behaviour is responsible directly or indirectly for the most common attacks (i.e., ransomware and phishing). To monitor user behaviour, it is necessary to process fast user logs coming from different and heterogeneous sources, having part of the data or some entire sources missing. A scalable framework based on the Elastic Stack (ELK) to process and store log data from different users and applications is proposed for this aim. This system generates an ensemble of models to classify user behaviour and detect anomalies in real-time. The scalability of the system is guaranteed by the ELK-based software architecture, running on top of a Kubernetes platform. In addition, a distributed evolutionary algorithm is used to classify the users by exploiting their digital footprints derived from many data sources. Experiments conducted on two real-life datasets verify the approach's goodness in detecting anomalies in the user behaviour, coping with missing data and lowering the number of false alarms.

show abstract

“…These aim to include mutually complementary individual classifiers, which are characterized by high diversity in terms of classifier structure [34], internal parameters or classifier inputs. As stated in [35], ensemble and meta-ensemble methods show a number of advantages with regard to using a single model, i.e., they reduce the variance of the error, the bias, and the dependence on a single dataset and work well in the case of unbalanced classes.…”

Section: The Proposed Modelmentioning

confidence: 99%

RDTIDS: Rules and Decision Tree-Based Intrusion Detection System for Internet-of-Things Networks

et al. 2020

View full text Add to dashboard Cite

This paper proposes a novel intrusion detection system (IDS), named RDTIDS, for Internet-of-Things (IoT) networks. The RDTIDS combines different classifier approaches which are based on decision tree and rules-based concepts, namely, REP Tree, JRip algorithm and Forest PA. Specifically, the first and second method take as inputs features of the data set, and classify the network traffic as Attack/Benign. The third classifier uses features of the initial data set in addition to the outputs of the first and the second classifier as inputs. The experimental results obtained by analyzing the proposed IDS using the CICIDS2017 dataset and BoT-IoT dataset, attest their superiority in terms of accuracy, detection rate, false alarm rate and time overhead as compared to state of the art existing schemes.

show abstract

Evolving meta-ensemble of classifiers for handling incomplete and unbalanced datasets in the cyber security domain

Cited by 27 publications

References 20 publications

Adaptive multiple imputations of missing values using the class center

Adaptive multiple imputations of missing values using the class center

A Scalable Cybersecurity Framework for Anomaly Detection in User Behaviour

RDTIDS: Rules and Decision Tree-Based Intrusion Detection System for Internet-of-Things Networks

Contact Info

Product

Resources

About