Model Reconstruction from Model Explanations

Milli, Smitha; Schmidt, Ludwig; Dragan, Anca D.; Hardt, Moritz

doi:10.1145/3287560.3287562

Cited by 116 publications

(108 citation statements)

References 15 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…Three organizations mentioned data privacy in the context of explainability, since in some cases explanations can be used to learn about the model [38,63] or the training data [56]. Methods to counter these concerns have been developed.…”

Section: On Privacymentioning

confidence: 99%

Explainable machine learning in deployment

Bhatt

Xiang

Sharma

et al. 2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

442

283

View full text Add to dashboard Cite

Section: On Privacymentioning

confidence: 99%

Explainable machine learning in deployment

Bhatt

Xiang

Sharma

et al. 2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

442

283

View full text Add to dashboard Cite

“…However, this method requires attackers to know the learning algorithm, the training data, etc. Milli et al [67] present an algorithm to learn a model through querying the gradient information of the target model for specific inputs. It is shown that gradient information can quickly reveal the model parameters.…”

Section: Model Extraction Attackmentioning

confidence: 99%

“…It is shown that gradient information can quickly reveal the model parameters. They conclude that gradient is a more efficient learning primitive than the predicted label [67]. However, this heuristic method introduces high computational overhead and they only evaluate their model extraction attacks on a two-layer neural network.…”

Section: Model Extraction Attackmentioning

confidence: 99%

Machine Learning Security: Threats, Countermeasures, and Evaluations

Xue

Yuan

Wu³

et al. 2020

IEEE Access

133

View full text Add to dashboard Cite

Machine learning has been pervasively used in a wide range of applications due to its technical breakthroughs in recent years. It has demonstrated significant success in dealing with various complex problems, and shows capabilities close to humans or even beyond humans. However, recent studies show that machine learning models are vulnerable to various attacks, which will compromise the security of the models themselves and the application systems. Moreover, such attacks are stealthy due to the unexplained nature of the deep learning models. In this survey, we systematically analyze the security issues of machine learning, focusing on existing attacks on machine learning systems, corresponding defenses or secure learning techniques, and security evaluation methods. Instead of focusing on one stage or one type of attack, this paper covers all the aspects of machine learning security from the training phase to the test phase. First, the machine learning model in the presence of adversaries is presented, and the reasons why machine learning can be attacked are analyzed. Then, the machine learning security-related issues are classified into five categories: training set poisoning; backdoors in the training set; adversarial example attacks; model theft; recovery of sensitive training data. The threat models, attack approaches, and defense techniques are analyzed systematically. To demonstrate that these threats are real concerns in the physical world, we also reviewed the attacks in real-world conditions. Several suggestions on security evaluations of machine learning systems are also provided. Last, future directions for machine learning security are also presented. INDEX TERMS Artificial intelligence security, poisoning attacks, backdoor attacks, adversarial examples, privacy-preserving machine learning.

show abstract

“…Similarly, network activations may be larger than the available on-chip memory and may be stored in RAM. These activations also need to be encrypted, because even in cases when the device manufacturer is not concerned about privacy, these activations can be used in order to infer the model weights [56].…”

Section: Attacks On Deployed Neural Networkmentioning

confidence: 99%

Survey of Attacks and Defenses on Edge-Deployed Neural Networks

Isakov¹,

Gadepally

Gettings

et al. 2019

2019 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

Deep Neural Network (DNN) workloads are quickly moving from datacenters onto edge devices, for latency, privacy, or energy reasons. While datacenter networks can be protected using conventional cybersecurity measures, edge neural networks bring a host of new security challenges. Unlike classic IoT applications, edge neural networks are typically very compute and memory intensive, their execution is data-independent, and they are robust to noise and faults. Neural network models may be very expensive to develop, and can potentially reveal information about the private data they were trained on, requiring special care in distribution. The hidden states and outputs of the network can also be used in reconstructing user inputs, potentially violating users' privacy. Furthermore, neural networks are vulnerable to adversarial attacks, which may cause misclassifications and violate the integrity of the output. These properties add challenges when securing edge-deployed DNNs, requiring new considerations, threat models, priorities, and approaches in securely and privately deploying DNNs to the edge. In this work, we cover the landscape of attacks on, and defenses, of neural networks deployed in edge devices and provide a taxonomy of attacks and defenses targeting edge DNNs.

show abstract

Model Reconstruction from Model Explanations

Cited by 116 publications

References 15 publications

Explainable machine learning in deployment

Explainable machine learning in deployment

Machine Learning Security: Threats, Countermeasures, and Evaluations

Survey of Attacks and Defenses on Edge-Deployed Neural Networks

Contact Info

Product

Resources

About