Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information

Abstract

Adversarial attacks against commercial black-box speech platforms, including cloud speech APIs and voice control devices, have received little attention until recent years. Constructing such attacks is difficult mainly due to the unique characteristics of time-domain speech signals and the much more complex architecture of acoustic systems. The current "black-box" attacks all heavily rely on the knowledge of prediction/confidence scores or other probability information to craft effective adversarial examples (AEs), which can be intuitively defended by service providers without returning these messages. In this paper, we take one more step forward and propose two novel adversarial attacks in more practical and rigorous scenarios. For commercial cloud speech APIs, we propose Occam, a decision-only black-box adversarial attack, where only final decisions are available to the adversary. In Occam, we formulate the decision-only AE generation as a discontinuous large-scale global optimization problem, and solve it by adaptively decomposing this complicated problem into a set of sub-problems and cooperatively optimizing each one. Our Occam is a one-size-fits-all approach, which achieves 100% success rates of attacks (SRoA) with an average SNR of 14.23dB, on a wide range of popular speech and speaker recognition APIs, including Google, Alibaba, Microsoft, Tencent, iFlytek, and Jingdong, outperforming the state-of-the-art black-box attacks. For commercial voice control devices, we propose NI-Occam, the first non-interactive physical adversarial attack, where the adversary does not need to query the oracle and has no access to its internal information and training data. We, for the first time, combine adversarial attacks with model inversion attacks, and thus generate the physically-effective audio AEs with high transferability without any interaction with target devices. Our experimental results show that NI-Occam can successfully fool Apple Siri, Microsoft Cortana, Google Assistant, iFlytek and Amazon Echo with an average SRoA of 52% and SNR of 9.65dB, shedding light on non-interactive physical attacks against voice control devices.

Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition

“…Securing commercial SRSs. We did not directly target commercial SRSs, although they are also vulnerable to black-box attacks [15], [73]. The reason is that it is more important to consider the most powerful adversaries when evaluating defenses, while the adversaries are not able to mount white-box attacks without having access to the internal structures of commercial SRSs.…”

“…FAKEBOB [15], SirenAttack [16], Kenansville [21], and Occam [73] are four black-box adversarial attacks targeting SRSs, where FAKEBOB, SirenAttack, and Occam are optimization-based attacks, and Kenansville is a signal processing-based attack. All of them, except for Occam which is not publicly available and non-trivial to reproduce, have been used to evaluate defenses in this work.…”

CommanderUAP: a practical and transferable universal adversarial attacks on speech recognition models

“…It was also the first SAP attack method to demonstrate effectiveness in the physical world. Chen et al (2020) proposed a method called Devil's Whisper, a black-box transfer attack on multiple commercial APIs and Intelligent Voice Control (IVC) devices, using substitute models. Other black-box attack methods, such as gradient estimation (Taori et al 2019), genetic algorithms (Alzantot et al 2018;Khare et al 2018;Du et al 2020), have also achieved high attack success rates.…”

“…We refer to this perturbation as a specific adversarial perturbation (SAP). Currently, the majority of work in this area is concentrated on SAP (Carlini and Wagner 2018;Qin et al 2019;Yuan et al 2018;Chen et al 2020;Khare et al 2018;Alzantot et al 2018;Du et al 2020;Taori et al 2019). In contrast, universal adversarial perturbation (UAP) is more destructive, which is audio-agnostic, meaning it has a high attack success rate when added to any audio example.…”

Adversarial Attack and Defense Strategies of Speaker Recognition Systems: A Survey

“…FakeBob proposed by Chen et al [81] has also been applied to black-box attacks, and the major approach is to transfer adversarial examples that have successfully attacked the grey-box system. Recently, CC-CMA-ES proposed by [91] applied a cooperative co-evolution (CC) framework to the powerful covariance matrix adaptation evolution strategy (CMA-ES) to solve the large and complex problem in the strictly black-box setting. Additionally, they adopt gradient inversion method to attack…”