Hear "No Evil", See "Kenansville": Efficient and Transferable Black-Box Attacks on Speech Recognition and Voice Identification Systems

Abstract

Automatic speech recognition and voice identification systems are being deployed in a wide array of applications, from providing control mechanisms to devices lacking traditional interfaces, to the automatic transcription of conversations and authentication of users. Many of these applications have significant security and privacy considerations. We develop attacks that force mistranscription and misidentification in state of the art systems, with minimal impact on human comprehension. Processing pipelines for modern systems are comprised of signal preprocessing and feature extraction steps, whose output is fed to a machine-learned model. Prior work has focused on the models, using white-box knowledge to tailor model-specific attacks. We focus on the pipeline stages before the models, which (unlike the models) are quite similar across systems. As such, our attacks are black-box and transferable, and demonstrably achieve mistranscription and misidentification rates as high as 100% by modifying only a few frames of audio. We perform a study via Amazon Mechanical Turk demonstrating that there is no statistically significant difference between human perception of regular and perturbed audio. Our findings suggest that models may learn aspects of speech that are generally not perceived by human subjects, but that are crucial for model accuracy. We also find that certain English language phonemes (in particular, vowels) are significantly more susceptible to our attack. We show that the attacks are effective when mounted over cellular networks, where signals are subject to degradation due to transcoding, jitter, and packet loss.

Can you trust what you hear: Effects of audio‐attacks on voice‐to‐face generation system

“…Meanwhile, some works have successfully provided adversarial samples on audio‐based systems with different settings and optimized objects. Such systems include speech recognition 19–22 and speaker recognition 23–26 . In the white‐box scenario, the process of generating adversarial samples using GD 17 can be simply defined as where is the gradient of with respect to .…”

“…For adversarial sample attacks, existing studies have largely focused on the space of images 17,18 . Some studies have successfully provided adversarial samples on audio‐based systems, including speech recognition 19–22 and speaker recognition 23–26 . The models used in the audio‐based systems are mainly composed of Time Delay Neural Network (TDNN), Long Short‐Term Memory (LSTM), or transformer encoders, which encode high‐dimensional temporal audio into low‐dimensional feature embedding.…”

“…17,18 Some studies have successfully provided adversarial samples on audio-based systems, including speech recognition [19][20][21][22] and speaker recognition. [23][24][25][26] The models used in the audio-based systems are mainly composed of Time Delay Neural Network (TDNN), Long Short-Term Memory (LSTM), or transformer encoders, which encode high-dimensional temporal audio into low-dimensional feature embedding. The adversarial audio causes these models to output the wrong word vector or the wrong speaker identity vector.…”

Sequential Randomized Smoothing for Adversarially Robust Speech Recognition

“…A few others specifically targeted audio inputs: the earliest was the ultrasonic-based DolphinAttack (Zhang et al, 2017) and the Houdini loss for structured models (Cisse et al, 2017), followed by the effective and popular Carlini&Wagner (CW) attack for audio . Other works have extented the state-of-the art with over-the air attacks (Yuan et al, 2018;Yakura and Sakuma, 2019; and black-box attacks that do not require gradient access and transfer well (Abdullah et al, 2021). A recent line of work has improved the imperceptibility of adversarial noise by using psychoacoustic models to constrain the noise rather than standard L 2 or L ∞ bounds Schönherr et al, 2019;Qin et al, 2019).…”

“…Provided with an input, these attacks will run gradient-based iterations to craft an additive noise. They are the hardest attacks to defend against, and a great metric to evaluate defenses that will carryover well to more practical attacks run over-the-air, without gradient access or in real time Abdullah et al, 2021). We consider two threat models.…”