Sponge Examples: Energy-Latency Attacks on Neural Networks

Shumailov, Ilia; Zhao, Yiren; Bates, Daniel; Papernot, Nicolas; Mullins, Robert; Anderson, Ross

doi:10.1109/eurosp51992.2021.00024

Cited by 63 publications

(42 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the exposure of the model's predictions represents a significant risk as an adversary can leverage this information to steal the model's knowledge [26,41,7,32,31,11,28,19]. The threat of such model extraction attacks is two-fold: adversaries may use the stolen model for monetary gains or as a reconnaissance step to mount further attacks [33,37].…”

Section: Introductionmentioning

confidence: 99%

Data-Free Model Extraction

Truong¹,

Maini²,

Walls³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Current model extraction attacks assume that the adversary has access to a surrogate dataset with characteristics similar to the proprietary data used to train the victim model. This requirement precludes the use of existing model extraction techniques on valuable models, such as those trained on rare or hard to acquire datasets. In contrast, we propose data-free model extraction methods that do not require a surrogate dataset. Our approach adapts techniques from the area of data-free knowledge transfer for model extraction. As part of our study, we identify that the choice of loss is critical to ensuring that the extracted model is an accurate replica of the victim model. Furthermore, we address difficulties arising from the adversary's limited access to the victim model in a black-box setting. For example, we recover the model's logits from its probability predictions to approximate gradients. We find that the proposed data-free model extraction approach achieves high-accuracy with reasonable query complexity -0.99× and 0.92× the victim model accuracy on SVHN and CIFAR-10 datasets given 2M and 20M queries respectively.

show abstract

Section: Introductionmentioning

confidence: 99%

Data-Free Model Extraction

Truong¹,

Maini²,

Walls³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Availability-based attacks have only recently gained the attention of researchers, despite the fact that a system's availability is a security-critical aspect of many applications. Shumailov et al [20] were the first to present an attack (called sponge examples) targeting the availability of computer vision and NLP models. In their paper, they demonstrated that adversarial examples are capable of doubling the inference time of NLP transformer-based models, with inference times ×6000 greater than regular input.…”

Section: Availability-based Attacksmentioning

confidence: 99%

“…In this paper, we present NMS-Sponge, the first availability attack against the end-to-end object detection pipeline, which is performed by applying a universal adversarial perturbation (UAP). Our initial attempt to apply the sponge attack proposed by Shumailov et al [20] on YOLO to decelerate inference processing was unsuccessful. This is due to the fact that for most of the images, the vast majority of the model's activation values are different than zero by default.…”

Section: Introductionmentioning

confidence: 99%

Denial-of-Service Attack on Object Detection Model Using Universal Adversarial Perturbation

Shapira¹,

Zolfi²,

Demetrio³

et al. 2022

Preprint

View full text Add to dashboard Cite

Adversarial attacks against deep learning-based object detectors have been studied extensively in the past few years. The proposed attacks aimed solely at compromising the models' integrity (i.e., trustworthiness of the model's prediction), while adversarial attacks targeting the models' availability, a critical aspect in safety-critical domains such as autonomous driving, have not been explored by the machine learning research community. In this paper, we propose NMS-Sponge, a novel approach that negatively affects the decision latency of YOLO, a state-ofthe-art object detector, and compromises the model's availability by applying a universal adversarial perturbation (UAP). In our experiments, we demonstrate that the proposed UAP is able to increase the processing time of individual frames by adding "phantom" objects while preserving the detection of the original objects.Recently, availability-based attacks have been shown to be effective against deep learning-based models. Shumailov et al. [20] presented sponge examples, which are perturbed inputs designed to increase the energy consumed by natural language processing (NLP) and computer vision models, when deployed on hardware accelerators, by increasing the number of active neurons during classification. Following this work, other studies have proposed sponge-like attacks, mainly targeting image classification models [4,3,6,10].Preprint. Under review.

show abstract

“…There are also poisoning attacks targeting DNNs' model availability. Shumailov et al [15] propose an attack that generates sponge examples, which significantly increase the energy consumption during model training. This leads to a longer run time during model inference, which negatively affects the model availability.…”

Section: Deep Learningmentioning

confidence: 99%

A Survey on Poisoning Attacks Against Supervised Machine Learning

Qiu¹

2022

Preprint

View full text Add to dashboard Cite

With the rise of artificial intelligence and machine learning in modern computing, one of the major concerns regarding such techniques is to provide privacy and security against adversaries. We present this survey paper to cover the most representative papers in poisoning attacks against supervised machine learning models. We first provide a taxonomy to categorize existing studies, and then present detailed summaries for selected papers. We summarize and compare the methodology and limitations of existing literature. We conclude this paper with potential improvements and future directions to further exploit and prevent poisoning attacks on supervised models. We propose several unanswered research questions to encourage and inspire researchers for future work.

show abstract

Sponge Examples: Energy-Latency Attacks on Neural Networks

Cited by 63 publications

References 52 publications

Data-Free Model Extraction

Data-Free Model Extraction

Denial-of-Service Attack on Object Detection Model Using Universal Adversarial Perturbation

A Survey on Poisoning Attacks Against Supervised Machine Learning

Contact Info

Product

Resources

About