Analysis of Deep Feature Loss based Enhancement for Speaker Verification

Abstract

Data augmentation is conventionally used to inject robustness in Speaker Verification systems. Several recently organized challenges focus on handling novel acoustic environments. Deep learning based speech enhancement is a modern solution for this. Recently, a study proposed to optimize the enhancement network in the activation space of a pre-trained auxiliary network. This methodology, called deep feature loss, greatly improved over the state-of-the-art conventional x-vector based system on a children speech dataset called BabyTrain. This work analyzes various facets of that approach and asks few novel questions in that context. We first search for optimal number of auxiliary network activations, training data, and enhancement feature dimension. Experiments reveal the importance of Signal-to-Noise Ratio filtering that we employ to create a large, clean, and naturalistic corpus for enhancement network training. To counter the "mismatch" problem in enhancement, we find enhancing front-end (x-vector network) data helpful while harmful for the back-end (Probabilistic Linear Discriminant Analysis (PLDA)). Importantly, we find enhanced signals contain complementary information to original. Established by combining them in front-end, this gives ~40% relative improvement over the baseline. We also do an ablation study to remove a noise class from x-vector data augmentation and, for such systems, we establish the utility of enhancement regardless of whether it has seen that noise class itself during training. Finally, we design several dereverberation schemes to conclude ineffectiveness of deep feature loss enhancement scheme for this task.

PL-EESR: Perceptual Loss Based END-TO-END Robust Speaker Representation Extraction

“…where the θ and the φ are the enhancement module and auxiliary speaker embedding extraction module respectively, and φ i denotes the i th layer of embedding extraction module. Equation ( 1) is what [17,18] used.…”

“…It was first proposed to deal with the image style transfer and super-resolution task [16]. This idea can be generalized for example to speaker verification task-specified enhancement training [17,18]. The approach used in [17,18] is similar to our method to compare the activation of enhanced and referenced signals using the auxiliary network.…”

“…This idea can be generalized for example to speaker verification task-specified enhancement training [17,18]. The approach used in [17,18] is similar to our method to compare the activation of enhanced and referenced signals using the auxiliary network. However, one drawback of [17,18] is that the auxiliary network is different from the speaker representation network although these two models are trained using the same task.…”

“…The approach used in [17,18] is similar to our method to compare the activation of enhanced and referenced signals using the auxiliary network. However, one drawback of [17,18] is that the auxiliary network is different from the speaker representation network although these two models are trained using the same task. The number of the total parameters in their system is up to 26.1M, which is larger than a standard x-vector network.…”

“…This also means that they adopt a two-stage training strategy and the enhancement model is independent of the speaker verification model. Another drawback is that the perceptual loss used in [17,18] relies heavily on clean speech as a training target which we need to select manually. In this paper, we propose an endto-end joint training network, of which the total number of parameters is 9.8M.…”