Machine Learning Security: Threats, Countermeasures, and Evaluations

Abstract

Machine learning has been pervasively used in a wide range of applications due to its technical breakthroughs in recent years. It has demonstrated significant success in dealing with various complex problems, and shows capabilities close to humans or even beyond humans. However, recent studies show that machine learning models are vulnerable to various attacks, which will compromise the security of the models themselves and the application systems. Moreover, such attacks are stealthy due to the unexplained nature of the deep learning models. In this survey, we systematically analyze the security issues of machine learning, focusing on existing attacks on machine learning systems, corresponding defenses or secure learning techniques, and security evaluation methods. Instead of focusing on one stage or one type of attack, this paper covers all the aspects of machine learning security from the training phase to the test phase. First, the machine learning model in the presence of adversaries is presented, and the reasons why machine learning can be attacked are analyzed. Then, the machine learning security-related issues are classified into five categories: training set poisoning; backdoors in the training set; adversarial example attacks; model theft; recovery of sensitive training data. The threat models, attack approaches, and defense techniques are analyzed systematically. To demonstrate that these threats are real concerns in the physical world, we also reviewed the attacks in real-world conditions. Several suggestions on security evaluations of machine learning systems are also provided. Last, future directions for machine learning security are also presented. INDEX TERMS Artificial intelligence security, poisoning attacks, backdoor attacks, adversarial examples, privacy-preserving machine learning.

Discovery Model Based on Analogies for Teaching Computer Programming

“…Despite the beginning of machine learning dating back several decades, it is currently considered as an emerging field for developing research processes [32], demonstrating unexpected results in complex situations that resemble processes developed by human experts or superior to them [33].…”

“…Machine learning generates learning with low computational complexity [34] by which it is possible to extract behavior patterns from a dataset and build predictive models [35] using two phases in data processing: training and testing [33]. Furthermore, the training data requires standardization processes to ensure its efficiency [36].…”

A Review of Machine Learning Algorithms for Cloud Computing Security

“…However, even with its unique characteristics, it has various security threats and protection challenges, as discussed in this section [30]. The categorization is performed based on the CIA Triad and attacks on cloud components [31,38].…”

A survey of machine learning techniques in adversarial image forensics

“…A number of literature surveys and reviews on the applications of ML-techniques in image forensics has been published in the literature ( Akhtar and Mian, 2018;Amodei et al, 2016;Papernot et al, 2016c;Xue et al, 2020 ;Ferreira et al, 2020;Kaur and Jindal, 2020;Verdoliva, 2020;Yang et al, 2020 ), although adversarial image forensics is generally not discussed. Amodei et al (2016) , for example, reviewed the general security concerns in artificial intelligence, particularly reinforcement learning and supervised learning algorithms.…”

“…Amodei et al (2016) , for example, reviewed the general security concerns in artificial intelligence, particularly reinforcement learning and supervised learning algorithms. A general review of security implications on the use of ML approaches and their countermeasures was presented by Papernot et al (2016c) , and Xue et al (2020) . Akhtar and Mian (2018) focused on adversarial attacks on deep learning approaches in computer vision.…”