Houdini: Fooling Deep Structured Prediction Models

Cissé, Moustapha; Adi, Yossi; Neverova, Natalia; Keshet, Joseph

doi:10.48550/arxiv.1707.05373

Cited by 63 publications

(96 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, knowing the architecture and parameters of a network could make easier for a malicious user to attack it, for instance with adversarial attacks. Indeed, if some black-box adversarial attacks do exist [54,50,13], many of them use the knowledge of the parameters of the network, at least to compute the gradients [56,21,28,41,10,39,38,5].…”

Section: Motivations: Privacy Robustness and Interpretabilitymentioning

confidence: 99%

See 1 more Smart Citation

Parameter identifiability of a deep feedforward ReLU neural network

Bona-Pellissier¹,

Bachoc²,

Malgouyres³

2021

Preprint

View full text Add to dashboard Cite

The possibility for one to recover the parameters -weights and biases-of a neural network thanks to the knowledge of its function on a subset of the input space can be, depending on the situation, a curse or a blessing. On one hand, recovering the parameters allows for better adversarial attacks and could also disclose sensitive information from the dataset used to construct the network. On the other hand, if the parameters of a network can be recovered, it guarantees the user that the features in the latent spaces can be interpreted. It also provides foundations to obtain formal guarantees on the performances of the network.It is therefore important to characterize the networks whose parameters can be identified and those whose parameters cannot.In this article, we provide a set of conditions on a deep fully-connected feedforward ReLU neural network under which the parameters of the network are uniquely identified -modulo permutation and positive rescaling-from the function it implements on a subset of the input space.

show abstract

Section: Motivations: Privacy Robustness and Interpretabilitymentioning

confidence: 99%

“…Equation (13) with λ k = ½ n k for all k ∈ 0, K is precisely equation ( 16). The reciprocal is clear: ( 16) is a particular case of ( 13) with λ k = ½ n k .…”

Section: • We Prove By Induction the Expression Of Fkmentioning

confidence: 99%

Parameter identifiability of a deep feedforward ReLU neural network

Bona-Pellissier¹,

Bachoc²,

Malgouyres³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition to high WER, the ASR model should predict a specific mistranscription target. Targeted attack is much more challenging [13,16] than untargeted attack.…”

Section: Introductionmentioning

confidence: 99%

Exploring Targeted Universal Adversarial Perturbations to End-to-end ASR Models

Lu¹,

Han²,

Cao³

2021

Preprint

View full text Add to dashboard Cite

Although end-to-end automatic speech recognition (e2e ASR) models are widely deployed in many applications, there have been very few studies to understand models' robustness against adversarial perturbations. In this paper, we explore whether a targeted universal perturbation vector exists for e2e ASR models. Our goal is to find perturbations that can mislead the models to predict the given targeted transcript such as "thank you" or empty string on any input utterance. We study two different attacks, namely additive and prepending perturbations, and their performances on the state-of-the-art LAS, CTC and RNN-T models. We find that LAS is the most vulnerable to perturbations among the three models. RNN-T is more robust against additive perturbations, especially on long utterances. And CTC is robust against both additive and prepending perturbations. To attack RNN-T, we find prepending perturbation is more effective than the additive perturbation, and can mislead the models to predict the same short target on utterances of arbitrary length.

show abstract

“…Yuan et al [10] used music as a carrier to hide speech commands in music. Cisse et al [11] proposed a more flexible attack method, which can be applied to different models. To attack the end-to-end ASR model, this method needed to obtain the loss of the target command and the current prediction result, and then find the adversarial example through optimization.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Example Devastation and Detection on Speech Recognition System by Adding Random Noise

Dong¹,

Yan²,

Gong³

et al. 2021

Preprint

View full text Add to dashboard Cite

The automatic speech recognition (ASR) system based on deep neural network is easy to be attacked by an adversarial example due to the vulnerability of neural network, which is a hot topic in recent years. The adversarial example does harm to the ASR system, especially if the commomdependent ASR goes wrong, it will lead to serious consequences. To improve the robustness and security of the ASR system, the defense method against adversarial examples must be proposed. Based on this idea, we propose an algorithm of devastation and detection on adversarial exsamples which can attack the current advanced ASR system. We choose advanced text-dependent and command-dependent ASR system as our target system. Generating adversarial examples by the OPT on text-dependent ASR and the GA-based algorithm on command-dependent ASR. The main idea of our method is input transformation of the adversarial examples. Different random intensities and kinds of noise are added to the adversarial examples to devastate the perturbation previously added to the normal examples. From the experimental results, the method performs well. For the devastation of examples, the original speech similarity before and after adding noise can reach 99.68%, the similarity of the adversarial examples can reach 0%, and the detection rate of the adversarial examples can reach 94%.

show abstract

Houdini: Fooling Deep Structured Prediction Models

Cited by 63 publications

References 26 publications

Parameter identifiability of a deep feedforward ReLU neural network

Parameter identifiability of a deep feedforward ReLU neural network

Exploring Targeted Universal Adversarial Perturbations to End-to-end ASR Models

Adversarial Example Devastation and Detection on Speech Recognition System by Adding Random Noise

Contact Info

Product

Resources

About