Robustness Evaluations of Sustainable Machine Learning Models against Data Poisoning Attacks in the Internet of Things

Abstract

With the increasing popularity of the Internet of Things (IoT) platforms, the cyber security of these platforms is a highly active area of research. One key technology underpinning smart IoT systems is machine learning, which classifies and predicts events from large-scale data in IoT networks. Machine learning is susceptible to cyber attacks, particularly data poisoning attacks that inject false data when training machine learning models. Data poisoning attacks degrade the performances of machine learning models. It is an ongoing research challenge to develop trustworthy machine learning models resilient and sustainable against data poisoning attacks in IoT networks. We studied the effects of data poisoning attacks on machine learning models, including the gradient boosting machine, random forest, naive Bayes, and feed-forward deep learning, to determine the levels to which the models should be trusted and said to be reliable in real-world IoT settings. In the training phase, a label modification function is developed to manipulate legitimate input classes. The function is employed at data poisoning rates of 5%, 10%, 20%, and 30% that allow the comparison of the poisoned models and display their performance degradations. The machine learning models have been evaluated using the ToN_IoT and UNSW NB-15 datasets, as they include a wide variety of recent legitimate and attack vectors. The experimental results revealed that the models’ performances will be degraded, in terms of accuracy and detection rates, if the number of the trained normal observations is not significantly larger than the poisoned data. At the rate of data poisoning of 30% or greater on input data, machine learning performances are significantly degraded.

Explainable artificial intelligence: a comprehensive review

“…The attacks mentioned above and the methods to defend against them have been studied extensively in recent years (Dunn et al 2020;Juuti et al 2019). The researchers have proved that they are precise and have excellent transferability, which has resulted in errors during the predictions of any AI model.…”

The Duo of Artificial Intelligence and Big Data for Industry 4.0: Review of Applications, Techniques, Challenges, and Future Research Directions

“…• Poisoning attacks: In these attack, adversaries aim to share manipulated labels such that the ML model would consider the re-training process to degrade the performance of the implemented model (assuming that adversaries either have complete control over the training dataset or can contribute to the training dataset [168]. • Evasion attacks: Unlike poisoning attacks, evasion attacks may occur after the model training phase, where the adversary may not know about the necessary data manipulation to attack the ML model.…”

Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspective

“…evasion attack vs. poisoning attack), and application scenarios, the suitable metrics may vary. For instance, in [32] the metrics Accuracy, Precision, False Positive, and True Positive were used to measure the negative impacts of the poisoning attacks to four ML models' integrity, including gradient boosted machines, random forests, naive Bayes statistical classifiers, and feed forward deep learning models, in the application of IoT environments; in [110], the evaluation was focused on the Recall metric to study the robustness of a Deep Learning-based network traffic classifier by applying several Universal Adversarial Perturbation based attacks against various traffic types including chat, email, file transfer, streaming, torrent, and VoIP; in [9], several metrics including Area Under the Curve (AUC, a type of false positive metric), Genuine Acceptance Rate (GAR) and False Acceptance Rate (FAR) were proposed to evaluate pattern classifiers used in adversarial applications like biometric authentication, network intrusion detection, and spam filtering.…”