Study of Multi-Class Classification Algorithms’ Performance on Highly Imbalanced Network Intrusion Datasets

Abstract

This paper is devoted to the problem of class imbalance in machine learning, focusing on the intrusion detection of rare classes in computer networks. The problem of class imbalance occurs when one class heavily outnumbers examples from the other classes. In this paper, we are particularly interested in classifiers, as pattern recognition and anomaly detection could be solved as a classification problem. As still a major part of data network traffic of any organization network is benign, and malignant traffic is rare, researchers therefore have to deal with a class imbalance problem. Substantial research has been undertaken in order to identify these methods or data features that allow to accurately identify these attacks. But the usual tactic to deal with the imbalance class problem is to label all malignant traffic as one class and then solve the binary classification problem. In this paper, however, we choose not to group or to drop rare classes but instead investigate what could be done in order to achieve good multi-class classification efficiency. Rare class records were up-sampled using SMOTE method (Chawla et al., 2002) to a preset ratio targets. Experiments with the 3 network traffic datasets, namely CIC-IDS2017, CSE-CIC-IDS2018 (Sharafaldin et al., 2018) and LITNET-2020(Damasevicius et al., 2020 were performed aiming to achieve reliable recognition of rare malignant classes available in these datasets.Popular machine learning algorithms were chosen for comparison of their readiness to support rare class detection. Related algorithm hyper parameters were tuned within a wide range of values, different data feature selection methods were used and tests were executed with and without oversampling to test the multiple class problem classification performance of rare classes.Machine learning algorithms ranking based on Precision, Balanced Accuracy Score,Ḡ, and prediction error Bias and Variance decomposition, show that decision tree ensembles (Adaboost, Random Forest Trees and Gradient Boosting Classifier) performed best on the network intrusion datasets used in this research.

GRU‐GBM: A combined intrusion detection model using LightGBM and gated recurrent unit

“…In Table 15, a comparison between the GRU‐GBM model and other models in the literature is presented. According to our results, GRU‐GBM achieved better accuracy than (Bulavas et al, 2021).…”

“…To deal with this problem, the random under‐sampling method is used for under‐sampling. We use skewed ratio function, where N is the number of initial records and s is the share of records in that class, to eliminate the imbalance problem (Bulavas et al, 2021). This formula provides that numbers of over‐represented classes are decreased by a non‐linear ratio while rare classes are left intact.…”

“…In the literature, there are only a few studies for multi-class classification tasks. LITNET-2020 dataset was evaluated in accordance with the dataset selection criteria (Gharib et al, 2017) by Bulavas et al (2021). It is one of the latest datasets with high compliance according to the abovementioned criteria.…”

“…It is one of the latest datasets with high compliance according to the abovementioned criteria. The researchers Bulavas et al (2021) firstly wanted to eliminate imbalanced distribution. Under‐sampling and over‐sampling were carried out with different classifiers such as Adaboost and Classification and Regression Tree (CART).…”

Discovery of new STAT3 inhibitors as anticancer agents using ligand-receptor contact fingerprints and docking-augmented machine learning

“… 19 However, we opted to remove closely similar docked poses/conformers to avoid noise leading to machine learning over-fitting errors. 67 Accordingly, any cluster of docked poses within RMSD ≤ 2.0 Å was represented by a single pose (of highest consensus score) in subsequent steps. Table 1 details the counts of docked poses before and after RMSD filtrations for the ionized docked ligands.…”