"Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World

Wenger, Emily; Bronckers, Max; Cianfarani, Christian; Cryan, Jenna; Sha, A.; Zheng, Haitao; Zhao, Ben Y.

doi:10.1145/3460120.3484742

Cited by 21 publications

(12 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, it is interesting to investigate whether input transformations can defend against those attacks. As a first attempt, we carry out a preliminary evaluation against hidden voice attack [78] and speech synthesis attack [79] (cf. Appendix A.8).…”

Section: Discussion Of Limitationsmentioning

confidence: 99%

“…[63], [80] for survey). There are other voice attacks in the speaker recognition domain, such as hidden voice attacks [78] and spoofing attacks [79], [88], [89], [90], [91], [92]. Though these attacks have different attack goals and scenarios from adversarial attacks [15], our preliminary evaluation shows that it is possible to mitigate hidden voice attack [78] and speech synthesis attack [79] via input transformations.…”

Section: Related Workmentioning

confidence: 94%

“…For speech synthesis attack, we exploit the deep learning-based speech synthesis tool used in [79]. The tool takes as input a set of voice samples of the source speaker and the desired speech content.…”

Section: A8 Defending Against Hidden Voice and Speech Synthesis Attacksmentioning

confidence: 99%

“…The tool takes as input a set of voice samples of the source speaker and the desired speech content. We use the voices in Spk10 enroll as the set of voice samples (10 speakers and 10 voices per speaker) and the ten sentences used in [79] as the desired speech content. We consider the following adaptive adversary: (1) running the non-adaptive speech synthesis attack with input voice samples x 1 , • • • , x N .…”

Section: A8 Defending Against Hidden Voice and Speech Synthesis Attacksmentioning

confidence: 99%

See 3 more Smart Citations

Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition

Chen¹,

Zhao²,

Fu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Speaker recognition systems (SRSs) have recently been shown to be vulnerable to adversarial attacks, raising significant security concerns. In this work, we systematically investigate transformation and adversarial training based defenses for securing SRSs. According to the characteristic of SRSs, we present 22 diverse transformations and thoroughly evaluate them using 7 recent promising adversarial attacks (4 white-box and 3 black-box) on speaker recognition. With careful regard for best practices in defense evaluations, we analyze the strength of transformations to withstand adaptive attacks. We also evaluate and understand their effectiveness against adaptive attacks when combined with adversarial training. Our study provides lots of useful insights and findings, many of them are new or inconsistent with the conclusions in the image and speech recognition domains, e.g., variable and constant bit rate speech compressions have different performance, and some non-differentiable transformations remain effective against current promising evasion techniques which often work well in the image domain. We demonstrate that the proposed novel feature-level transformation combined with adversarial training is rather effective compared to the sole adversarial training in a complete white-box setting, e.g., increasing the accuracy by 13.62% and attack cost by two orders of magnitude, while other transformations do not necessarily improve the overall defense capability. This work sheds further light on the research directions in this field. We also release our evaluation platform SPEAKERGUARD to foster further research.

show abstract

Section: Discussion Of Limitationsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 94%

Section: A8 Defending Against Hidden Voice and Speech Synthesis Attacksmentioning

confidence: 99%

Section: A8 Defending Against Hidden Voice and Speech Synthesis Attacksmentioning

confidence: 99%

See 2 more Smart Citations

Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition

Chen¹,

Zhao²,

Fu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A successful low-resource attack can be trained with approximately 2 minutes of data. Confirming the results in [110], these systems lack the element of verifying the data source validity, thus, there is an urgent need to design solutions that check the integrity of the data before blocking it.…”

Section: Attacking Commercial Voicementioning

confidence: 99%

Locally Authenticated Privacy-preserving Voice Input

Aloufi¹,

Nautsch²,

Haddadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Increasing use of our biometrics (e.g., fingerprints, faces, or voices) to unlock access to and interact with online services raises concerns about the trade-offs between convenience, privacy, and security. Service providers must authenticate their users, although individuals may wish to maintain privacy and limit the disclosure of sensitive attributes beyond the authentication step, e.g., when interacting with Voice User Interfaces (VUIs). Preserving privacy while performing authentication is challenging, particularly where adversaries can use biometric data to train transformation tools (e.g., 'deepfaked' speech) and use the faked output to defeat existing authentication systems. In this paper, we take a step towards understanding security and privacy requirements to establish the threat and defense boundaries. We introduce a secure, flexible privacypreserving system to capture and store an on-device fingerprint of the users' raw signals (i.e., voice) for authentication instead of sending/sharing the raw biometric signals. We then analyze this fingerprint using different predictors, each evaluating its legitimacy from a different perspective (e.g., target identity claim, spoofing attempt, and liveness). We fuse multiple predictors' decisions to make a final decision on whether the user input is legitimate or not. Validating legitimate users yields an accuracy rate of 98.68% after cross-validation using our verification technique. The pipeline runs in tens of milliseconds when tested on a CPU and a single-core ARM processor, without specialized hardware.

show abstract