Fuzzy–synthetic minority oversampling technique: Oversampling based on fuzzy set theory for Android malware detection in imbalanced datasets

Abstract

In previous work, imbalanced datasets composed of more benign samples (the majority class) than the malicious one (the minority class) have been widely adopted in Android malware detection. These imbalanced datasets bias learning toward the majority class, so that the minority class examples are more likely to be misclassified. To solve the problem, we propose a new oversampling method called fuzzy-synthetic minority oversampling technique, which is based on fuzzy set theory and the synthetic minority oversampling technique method. As the sample size of the majority class increases relative to that of the minority class, fuzzy-synthetic minority oversampling technique generates more synthetic examples for each minority class examples in the fuzzy region, where the minority examples have a low degree of membership to the minority class and are more likely to be misclassified. Using the new synthetic examples, the classifiers build larger decision regions that contain more minority examples, and they are no longer biased to the majority class. Compared with synthetic minority oversampling technique and Borderline-synthetic minority oversampling technique methods, fuzzy-synthetic minority oversampling technique achieves higher accuracy on both the minority class and the entire datasets.

Oversampling Imbalanced Data Based on Convergent WGAN for Network Threat Detection

“…Next, the generator is to approximately minimize the Wasserstein distance, which is equivalent to minimize L in formula (5). Considering that the first term of formula ( 5) is independent of the generator, we can get the discriminator loss and generator loss of WGAN in formulas ( 6) and (7).…”

“…Formula ( 6) is the inverse of formula (5), and formula ( 6) can indicate the training process. e smaller the value of formula ( 6) is, the smaller the Wasserstein distance between the real data and the generated data is, and the better WGAN training is.…”

“…For example, in network intrusion detection, most traffic issues are normal with very few abnormal ones [4]. In Android malware detection, the proportion of the malware APPs is relatively low [5]. In the network threat detection, if the malicious samples as minority class are misjudged and widely spread, it will bring big losses to users.…”

Synthetic Minority Oversampling Technique for Optimizing Classification Tasks in Botnet and Intrusion-Detection-System Datasets

“…Similarly, in [33], random sampling fixed an offset between the number of minority classes by random generation, enabling a better detection ratio. The authors in [34] discussed that the repeated production and mixing of oversampled instances can create some disturbance in the original data distribution; instead, a series of SMOTE-based adaptations were adopted: In [10], the authors said that malware-based datasets are for the most part constituted by benign samples, requiring a feasible structure that considers the degree of the belongingness of each minority class, a decisive factor to properly balance a dataset. As part of the methodology, a set of algorithms (DT, RF, Naive Bayes-NB, SVM, and AdaBoost-AB) were trough-fed preprocessed inputs under a fuzzy-theory-based SMOTE technique, reducing misclassification costs to a large extent [35].…”

“…In supervised ML-based problems, when a certain number of classes are not equally distributed, the data are said to be unbalanced, impacting the algorithm capabilities to aptly learn from the samples of the predominant class, following to a degradation of classification performance. Considering [9,10], the credibility of an ML-based cyber-physical system depends on the predictive abilities of intelligent agents trained with a wide range of adversarial behavioral patterns, able to protect workloads against malicious activities. Indeed, data distribution has a great effect on the efficiency of different ML models [11,12]; thus, it is important to establish data-level strategies in the preprocessing and training stages.…”