Sponge Examples: Energy-Latency Attacks on Neural Networks

Shumailov, Ilia; Zhao, Yiren; Bates, Daniel J.; Papernot, Nicolas; Mullins, Robert F.; Anderson, Ross

doi:10.48550/arxiv.2006.03463

Cited by 10 publications

(13 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Integrity of ML systems can be compromised by adversarial examples [Biggio et al 2013;Szegedy et al 2014] and data poisoning attacks [Biggio et al 2012;]. Finally, resource-depletion attacks [Shumailov et al 2020;Hong et al 2020a] can threaten the Availability of ML systems.…”

Section: Data Hub Solutionmentioning

confidence: 99%

On the Opportunities and Risks of Foundation Models

Bommasani¹,

Hudson²,

Adeli³

et al. 2021

Preprint

649

652

View full text Add to dashboard Cite

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

show abstract

Section: Data Hub Solutionmentioning

confidence: 99%

On the Opportunities and Risks of Foundation Models

Bommasani¹,

Hudson²,

Adeli³

et al. 2021

Preprint

649

652

View full text Add to dashboard Cite

show abstract

“…Several security threats are studied regarding machine learning focusing on the basic information security triad: confidentiality, integrity, and availability. For instance (Shumailov et al, 2020) present an ML attack targeting the model's availability. According to (He et al, 2019) the main attack categories for integrity are adversarial and poisoning attacks, while for confidentiality, these are model extraction and model inversion.…”

Section: Related Workmentioning

confidence: 99%

Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexity

Parisot¹,

Pejó²

2021

Preprint

View full text Add to dashboard Cite

Machine learning models' goal is to make correct predictions for specific tasks by learning important properties and patterns from data. By doing so, there is a chance that the model learns properties that are unrelated to its primary task. Property Inference Attacks exploit this and aim to infer from a given model (i.e., the target model) properties about the training dataset seemingly unrelated to the model's primary goal. If the training data is sensitive, such an attack could lead to privacy leakage. In this paper, we investigate the influence of the target model's complexity on the accuracy of this type of attack, focusing on convolutional neural network classifiers. We perform attacks on models that are trained on facial images to predict whether someone's mouth is open. Our attacks' goal is to infer whether the training dataset is balanced genderwise. Our findings reveal that the risk of a privacy breach is present independently of the target model's complexity: for all studied architectures, the attack's accuracy is clearly over the baseline. We discuss the implication of the property inference on personal data in the light of Data Protection Regulations and Guidelines.

show abstract

“…In model reverse engineering, crafted inputs allow to deduce from a trained model whether dropout was used and other architectural choices [63]. Finally, sponge attacks aim to increase energy consumption of the classifier at test time [72].…”

Section: Adversarial Machine Learningmentioning

confidence: 99%

“…During our interviews, we found evidence that semiautomated fraud on ML systems takes place in the wild. Our findings on mental models allow to tackle these threats by (I) aligning corporate workflows that enable all actors to understand AML threats with minimal effort, (II) developing tools data design training model deployment poisoning [53,67], backdooring [19,36] model stealing [82] model reverse engineering [63] membership inference [71] evasion [23,78] adversarial reprogramming [27] adversarial initialization [32,52] weight perturbations [20] sponge attacks [72] Figure 1: AML threats within the ML pipeline. Each attack is visualized as an arrow pointing from the step controlled to the point where the attack affects the pipeline.…”

Section: Introductionmentioning

confidence: 99%

Mental Models of Adversarial Machine Learning

Bieringer¹,

Grosse²,

Backes³

et al. 2021

Preprint

View full text Add to dashboard Cite

Although machine learning (ML) is widely used in practice, little is known about practitioners' actual understanding of potential security challenges. In this work, we close this substantial gap in the literature and contribute a qualitative study focusing on developers' mental models of the ML pipeline and potentially vulnerable components. Studying mental models has helped in other security fields to discover root causes or improve risk communication. Our study reveals four characteristic ranges in mental models of industrial practitioners. The first range concerns the intertwined relationship of adversarial machine learning (AML) and classical security. The second range describes structural and functional components. The third range expresses individual variations of mental models, which are neither explained by the application nor by the educational background of the corresponding subjects. The fourth range corresponds to the varying levels of technical depth, which are however not determined by our subjects' level of knowledge. Our characteristic ranges have implications for the integration of AML into corporate workflows, security enhancing tools for practitioners, and creating appropriate regulatory frameworks for AML.

show abstract

Sponge Examples: Energy-Latency Attacks on Neural Networks

Cited by 10 publications

References 43 publications

On the Opportunities and Risks of Foundation Models

On the Opportunities and Risks of Foundation Models

Property Inference Attacks on Convolutional Neural Networks: Influence and Implications of Target Model's Complexity

Mental Models of Adversarial Machine Learning

Contact Info

Product

Resources

About