Feature Selection Improves Tree-based Classification for Wireless Intrusion Detection

Abstract

With the growth of 5G wireless technologies and IoT, it become urgent to develop robust network security systems, such as intrusions detection systems (IDS) to keep the networks secure. These IDS systems need to detect unauthorized access and attacks in real-time. However, most of the modern IDS are built based on complex machine learning models that are time-consuming to train. In this work, we propose a methodology using the SHapley Additive exPlanations (SHAP) in combination with tree-based classifiers. SHAP can be used to select consistent and small feature subsets to reduce the execution time and improve classification accuracy. We demonstrate the proposed approach with the Aegean Wi-Fi Intrusion Dataset (AWID) dataset in a series of multi-class classification experiments. Among the four classes ("normal", "injection", "flooding" and "impersonation"), it is well-known that the class impersonation is hard to be classified accurately. Tests show that we can use about 10% of the initial feature set without reducing the overall prediction accuracy. With this reduced set of features, the training time could be reduced as much as a factor of four, while slightly improving the discriminating ability to identify impersonation instances. This study suggests that by reducing the number of features, the classification algorithms are able to focus on key trends that differentiates the "attacks" classes from the "normal" class. Using a reduces subset of features improves IDS's accuracy and performance. Also, SHAP dependence plots capture the relationship between individual features and the classification decision.

Pick Quality Over Quantity: Expert Feature Selection and Data Preprocessing for 802.11 Intrusion Detection Systems

“…The authors in [5] perform feature selection as a first step for tree-based classification of normal and malicious wireless traffic. Initially, they removed features that have zero values for at least 99% of the instances.…”

“…Additionally, some contributions mention evaluation metric scores that seem questionable. For instance, each one of the works in [5,9,13,12] presents an almost equal Precision, Recall, F1, and Accuracy scores. Based on these results, it can be assumed that these works most probably employ a balanced dataset, and indeed this is the case with [5].…”

“…For instance, each one of the works in [5,9,13,12] presents an almost equal Precision, Recall, F1, and Accuracy scores. Based on these results, it can be assumed that these works most probably employ a balanced dataset, and indeed this is the case with [5]. Nevertheless, in real-world scenarios, the wireless traffic will be, in all likelihood, imbalanced.…”

“…However, they never received wide adoption or the development of some of them has been abandoned. On the other hand, a critical mass of works [4,5,6] deals with the application of Machine Learning (ML) techniques for the development of misuse detection tools that are optimized for the detection of attacks in 802.11 wireless networks.…”

GPU-based Classification for Wireless Intrusion Detection

“…Using the same AWID dataset, Bhandari et al [1] showed that by selecting a subset of important features of size 10% of the initial set, the training time can be reduced by 4x, while improving the discriminating ability to identify attack instances. This study employed the SHAP method for feature selection and concluded that by selecting a small subset of the most important features, the algorithm is able to important patterns to discriminate between the "attack" instances and the "normal" instances.…”