Improving on-device speaker verification using federated learning with privacy

Granqvist, Filip; Seigel, Matt; Rogier, van Dalen,; Cahill, Áine; Shum, Stephen; Paulik, Matthias

doi:10.48550/arxiv.2008.02651

Cited by 11 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, federated systems still leak information via model updates [110]. Recent work by Granqvist et al [111] therefore combine federated learning with differential privacy. With differential privacy, noise is added to model updates to provide a guaranteed upper bound on the amount of information that can be leaked.…”

Section: A Privacy Preservationmentioning

confidence: 99%

Personal Voice Assistant Security and Privacy—A Survey

Cheng

Roedig

2022

Proc. IEEE

View full text Add to dashboard Cite

Personal voice assistants (PVAs) are increasingly used as interfaces to digital environments. Voice commands are used to interact with phones, smart homes, or cars. In the United States alone, the number of smart speakers, such as Amazon's Echo and Google Home, has grown by 78% to 118.5 million, and 21% of the U.S. population own at least one device. Given the increasing dependency of society on PVAs, security and privacy of these have become a major concern of users, manufacturers, and policy makers. Consequently, a steep increase in research efforts addressing security and privacy of PVAs can be observed in recent years. While some security and privacy research applicable to the PVA domain predates their recent increase in popularity, many new research strands have emerged. This article provides a survey of the state of the art in PVA security and privacy. The focus of this work is on the security and privacy challenges arising from the use of the acoustic channel. Work that describes both attacks and countermeasures is discussed. We highlight established areas such as voice authentication (VA) and new areas such as acoustic Denial of Service (DoS) that deserve more attention. This survey describes research areas where the threat is relatively well understood but where countermeasures are lacking, for example, in the area of hidden voice commands. We also discuss work that looks at privacy implications; for example, work on management of recording Manuscript

show abstract

Section: A Privacy Preservationmentioning

confidence: 99%

Personal Voice Assistant Security and Privacy—A Survey

Cheng

Roedig

2022

Proc. IEEE

View full text Add to dashboard Cite

show abstract

“…On-device-based Methods. Some works suggest running the systems on the device [35,97] and designing light encryption systems. Im et al in [43] propose a user-friendly, privacy-preserving face authentication system for smartphones to prevent malicious users from accessing the system.…”

Section: Secure Privacy-aware Biometric Systemsmentioning

confidence: 99%

Locally Authenticated Privacy-preserving Voice Input

Aloufi¹,

Nautsch²,

Haddadi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Increasing use of our biometrics (e.g., fingerprints, faces, or voices) to unlock access to and interact with online services raises concerns about the trade-offs between convenience, privacy, and security. Service providers must authenticate their users, although individuals may wish to maintain privacy and limit the disclosure of sensitive attributes beyond the authentication step, e.g., when interacting with Voice User Interfaces (VUIs). Preserving privacy while performing authentication is challenging, particularly where adversaries can use biometric data to train transformation tools (e.g., 'deepfaked' speech) and use the faked output to defeat existing authentication systems. In this paper, we take a step towards understanding security and privacy requirements to establish the threat and defense boundaries. We introduce a secure, flexible privacypreserving system to capture and store an on-device fingerprint of the users' raw signals (i.e., voice) for authentication instead of sending/sharing the raw biometric signals. We then analyze this fingerprint using different predictors, each evaluating its legitimacy from a different perspective (e.g., target identity claim, spoofing attempt, and liveness). We fuse multiple predictors' decisions to make a final decision on whether the user input is legitimate or not. Validating legitimate users yields an accuracy rate of 98.68% after cross-validation using our verification technique. The pipeline runs in tens of milliseconds when tested on a CPU and a single-core ARM processor, without specialized hardware.

show abstract

“…Coupled with differential privacy, it can allow learning of a model with strong privacy guarantees. Models trained via private federated learning have successfully improved existing on-device applications while preserving users' privacy [16,17,32]. This has led to a lot of ongoing research on designing better algorithms for federated learning applications.…”

Section: Introductionmentioning

confidence: 99%

FLAIR: Federated Learning Annotated Image Repository

Song¹,

Granqvist²,

Talwar³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Cross-device federated learning is an emerging machine learning (ML) paradigm where a large population of devices collectively train an ML model while the data remains on the devices. This research field has a unique set of practical challenges, and to systematically make advances, new datasets curated to be compatible with this paradigm are needed. Existing federated learning benchmarks in the image domain do not accurately capture the scale and heterogeneity of many real-world use cases. We introduce FLAIR, a challenging large-scale annotated image dataset for multi-label classification suitable for federated learning. FLAIR has 429,078 images from 51,414 Flickr users and captures many of the intricacies typically encountered in federated learning, such as heterogeneous user data and a long-tailed label distribution. We implement multiple baselines in different learning setups for different tasks on this dataset. We believe FLAIR can serve as a challenging benchmark for advancing the state-of-the art in federated learning. Dataset access and the code for the benchmark are available at https://github.com/apple/ ml-flair. * Equal contribution Preprint. Under review.

show abstract

Improving on-device speaker verification using federated learning with privacy

Cited by 11 publications

References 0 publications

Personal Voice Assistant Security and Privacy—A Survey

Personal Voice Assistant Security and Privacy—A Survey

Locally Authenticated Privacy-preserving Voice Input

FLAIR: Federated Learning Annotated Image Repository

Contact Info

Product

Resources

About