Training Keyword Spotting Models on Non-IID Data with Federated Learning

Hard, Andrew; Partridge, Kurt; Nguyen, Cameron; Subrahmanya, Niranjan; Shah, Aishanee; Zhu, Pai; Moreno, Ignacio López; Mathews, Rajiv

doi:10.21437/interspeech.2020-3023

Cited by 42 publications

(20 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When Training on non-IID data has been shown to be sub-optimal across multiple domains [12,13,14] and remains an open problem in federated learning [15,16].…”

Section: Introductionmentioning

confidence: 99%

Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Guliani

Beaufays

Motta

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We propose using federated learning, a decentralized ondevice learning paradigm, to train speech recognition models. By performing epochs of training on a per-user basis, federated learning must incur the cost of dealing with non-IID data distributions, which are expected to negatively affect the quality of the trained model. We propose a framework by which the degree of non-IID-ness can be varied, consequently illustrating a trade-off between model quality and the computational cost of federated training, which we capture through a novel metric. Finally, we demonstrate that hyperparameter optimization and appropriate use of variational noise are sufficient to compensate for the quality impact of non-IID distributions, while decreasing the cost.

show abstract

“…When Training on non-IID data has been shown to be sub-optimal across multiple domains [12,13,14] and remains an open problem in federated learning [15,16].…”

Section: Introductionmentioning

confidence: 99%

Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Guliani

Beaufays

Motta

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…In [6], federated learning was used to train an embedded wake word detector on crowdsourced speech. In [7], various federated averaging schemes and data augmentation techniques have been studied to improve keyword spotting models with data not independent and identically distributed (iid) at the edge. An interactive system was built in [8] to demonstrate how FL can help transfer learning on acoustic models using edge device data.…”

Section: Introductionmentioning

confidence: 99%

Federated Acoustic Modeling for Automatic Speech Recognition

Cui

Kingsbury

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Data privacy and protection is a crucial issue for any automatic speech recognition (ASR) service provider when dealing with clients. In this paper, we investigate federated acoustic modeling using data from multiple clients. A client's data is stored on a local data server and the clients communicate only model parameters with a central server, and not their data. The communication happens infrequently to reduce the communication cost. To mitigate the non-iid issue, client adaptive federated training (CAFT) is proposed to canonicalize data across clients. The experiments are carried out on 1,150 hours of speech data from multiple domains. Hybrid LSTM acoustic models are trained via federated learning and their performance is compared to traditional centralized acoustic model training. The experimental results demonstrate the effectiveness of the proposed federated acoustic modeling strategy. We also show that CAFT can further improve the performance of the federated acoustic model.

show abstract

“…In recent years, federated learning was applied successfully in many different tasks. In [9,10], federated learning is used in wake-word detection for a digital assistant. Federated learning was also applied in speech emotion detection [11].…”

Section: Introductionmentioning

confidence: 99%

Federated Learning in ASR: Not as Easy as You Think

Yu,

Freiwald,

Tewes

et al. 2021

Preprint

View full text Add to dashboard Cite

With the growing availability of smart devices and cloud services, personal speech assistance systems are increasingly used on a daily basis. Most devices redirect the voice recordings to a central server, which uses them for upgrading the recognizer model. This leads to major privacy concerns, since private data could be misused by the server or third parties. Federated learning is a decentralized optimization strategy that has been proposed to address such concerns. Utilizing this approach, private data is used for on-device training. Afterwards, updated model parameters are sent to the server to improve the global model, which is redistributed to the clients. In this work, we implement federated learning for speech recognition in a hybrid and an end-toend model. We discuss the outcomes of these systems, which both show great similarities and only small improvements, pointing to a need for a deeper understanding of federated learning for speech recognition.

show abstract

Training Keyword Spotting Models on Non-IID Data with Federated Learning

Cited by 42 publications

References 0 publications

Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Federated Acoustic Modeling for Automatic Speech Recognition

Federated Learning in ASR: Not as Easy as You Think

Contact Info

Product

Resources

About