On the Security Relevance of Initial Weights in Deep Neural Networks

Grosse, Kathrin; Trost, Thomas Alexander; Mosbach, Marius; Backes, Michael; Klakow, Dietrich

doi:10.1007/978-3-030-61609-0_1

Cited by 4 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the sake of completeness, we conclude with a description of additional, recent attacks, some of which are part of our questionnaires (Appendix D.3). In adversarial initialization, the initial weights of a neural network 1 are targeted to harm convergence or accuracy during training [32,52]. In adversarial reprogramming, an input perturbation mask forces the classifier at test time to perform another classification task than originally intended [27].…”

Section: Adversarial Machine Learningmentioning

confidence: 99%

“…We also investigated the familiarity of our subjects with AML attacks.To avoid priming, we asked subjects to rate their familiarity after the interview. As sanity checks, we added two rather unknown terms, adversarial initialization [32] and neural trojans [54] (similar to backdoors). The results are depicted in Figure 8.…”

Section: B Subjects Prior Knowledge On Amlmentioning

confidence: 99%

“…During our interviews, we found evidence that semiautomated fraud on ML systems takes place in the wild. Our findings on mental models allow to tackle these threats by (I) aligning corporate workflows that enable all actors to understand AML threats with minimal effort, (II) developing tools data design training model deployment poisoning [53,67], backdooring [19,36] model stealing [82] model reverse engineering [63] membership inference [71] evasion [23,78] adversarial reprogramming [27] adversarial initialization [32,52] weight perturbations [20] sponge attacks [72] Figure 1: AML threats within the ML pipeline. Each attack is visualized as an arrow pointing from the step controlled to the point where the attack affects the pipeline.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mental Models of Adversarial Machine Learning

Bieringer¹,

Grosse²,

Backes³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Although machine learning (ML) is widely used in practice, little is known about practitioners' actual understanding of potential security challenges. In this work, we close this substantial gap in the literature and contribute a qualitative study focusing on developers' mental models of the ML pipeline and potentially vulnerable components. Studying mental models has helped in other security fields to discover root causes or improve risk communication. Our study reveals four characteristic ranges in mental models of industrial practitioners. The first range concerns the intertwined relationship of adversarial machine learning (AML) and classical security. The second range describes structural and functional components. The third range expresses individual variations of mental models, which are neither explained by the application nor by the educational background of the corresponding subjects. The fourth range corresponds to the varying levels of technical depth, which are however not determined by our subjects' level of knowledge. Our characteristic ranges have implications for the integration of AML into corporate workflows, security enhancing tools for practitioners, and creating appropriate regulatory frameworks for AML.

show abstract

Section: Adversarial Machine Learningmentioning

confidence: 99%

Section: B Subjects Prior Knowledge On Amlmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Mental Models of Adversarial Machine Learning

Bieringer¹,

Grosse²,

Backes³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

On the Security Relevance of Initial Weights in Deep Neural Networks

Grosse

Trost

Mosbach

et al. 2020

Artificial Neural Networks and Machine Learning – ICANN 2020

View full text Add to dashboard Cite

Algorithmic and Implementation-Based Threats for the Security of Embedded Machine Learning Models

Moëllic,

Dumont,

Hector

et al. 2024

Studies in Computational Intelligence

View full text Add to dashboard Cite

The large-scale deployment of machine learning models in a wide variety of AI-based systems raises major security concerns related to their integrity, confidentiality and availability. These security issues encompass the overall traditional machine learning pipeline, including the training and the inference processes. In the case of embedded models deployed in physically accessible devices, the attack surface is particularly complex because of additional attack vectors exploiting implementation-based flaws. This chapter aims at describing the most important attacks that threaten state-of-the-art embedded machine learning models (especially deep neural networks) widely deployed in IoT applications (e.g., health, industry, transport) and highlighting new critical attack vectors that rely on side-channel and fault injection analysis and significantly extend the attack surface of AIoT systems (Artificial Intelligence of Things). More particularly, we focus on two advanced threats against models deployed in 32-bit microcontrollers: model extraction and weight-based adversarial attacks.

show abstract

On the Security Relevance of Initial Weights in Deep Neural Networks

Cited by 4 publications

References 18 publications

Mental Models of Adversarial Machine Learning

Mental Models of Adversarial Machine Learning

On the Security Relevance of Initial Weights in Deep Neural Networks

Algorithmic and Implementation-Based Threats for the Security of Embedded Machine Learning Models

Contact Info

Product

Resources

About