Within-Sample Variability-Invariant Loss for Robust Speaker Recognition Under Noisy Environments

Abstract

Despite the significant improvements in speaker recognition enabled by deep neural networks, unsatisfactory performance persists under noisy environments. In this paper, we train the speaker embedding network to learn the "clean" embedding of the noisy utterance. Specifically, the network is trained with the original speaker identification loss with an auxiliary within-sample variability-invariant loss. This auxiliary variability-invariant loss is used to learn the same embedding among the clean utterance and its noisy copies and prevents the network from encoding the undesired noises or variabilities into the speaker representation. Furthermore, we investigate the data preparation strategy for generating clean and noisy utterance pairs on-the-fly. The strategy generates different noisy copies for the same clean utterance at each training step, helping the speaker embedding network generalize better under noisy environments. Experiments on VoxCeleb1 indicate that the proposed training framework improves the performance of the speaker verification system in both clean and noisy conditions.

An Iterative Framework for Self-Supervised Deep Speaker Representation Learning

“…Data augmentation is proven to be an effective strategy for both conventional learning with supervision [24] and contrastive selfsupervision learning [7,8,15] in the context of deep learning. We perform data augmentation with MUSAN dataset [25].…”

“…We use the same network architecture as in [24]. ReLU nonlinear activation and batch normalization are applied to each convolutional layer in ResNet.…”

Contrastive Self-Supervised Learning for Text-Independent Speaker Verification

“…Channel-invariant training. Inspired by [19], we propose channel-invariant loss which is used to force the embedding of augmented segments as similar as its clean version, preventing the network from encoding the undesired channel information into the speaker representation. The model can learn to filter out channel factors.…”

DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning

“…For input-level, models usually can be adapted by training with enhanced [8] or domain-translated [9] input features. For adaptation at embedding-level, it often targets at minimizing certain distances between source and target domains to align them in the same embedding space, such as cosine distance [10], mean squared error (MSE) [11], and maximum mean discrepancy (MMD) [12]. However, this method usually requires parallel or artificial simulated data, which cannot generalize well to real-world scenarios.…”