Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System

Abstract

In this paper, we explore the encoding/pooling layer and loss function in the end-to-end speaker and language recognition system. First, a unified and interpretable end-to-end system for both speaker and language recognition is developed. It accepts variable-length input and produces an utterance level result. In the end-to-end system, the encoding layer plays a role in aggregating the variable-length input sequence into an utterance level representation. Besides the basic temporal average pooling, we introduce a self-attentive pooling layer and a learnable dictionary encoding layer to get the utterance level representation. In terms of loss function for open-set speaker verification, to get more discriminative speaker embedding, center loss and angular softmax loss is introduced in the end-to-end system. Experimental results on Voxceleb and NIST LRE 07 datasets show that the performance of end-to-end learning system could be significantly improved by the proposed encoding layer and loss function.arXiv:1804.05160v1 [eess.AS]

Within-Sample Variability-Invariant Loss for Robust Speaker Recognition Under Noisy Environments

“…An encoding layer is then applied to the top of it to get the utterance level representation. The most common encoding method is the average pooling layer, which aggregates the statistics (i.e., mean, or mean and standard deviation) [1,2].…”

HI-MIA: A Far-Field Text-Dependent Speaker Verification Database and the Baselines

“…The superiority of deep speaker embedding systems has been shown in text-independent speaker recognition for closed talking [21,22] and far-field scenarios [24,25]. In this paper, we Figure 2: Gender and age distribution adopt the deep speaker embedding system, which is initially designed for the text-independent speaker verification, as baseline.…”

“…The single-channel network structure is the same as in [22]. There are three main components in this framework.…”

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

“…One aspect of our study is therefore an attempt to find out how effective these recent developments in speaker verification are for speaker adaption in TTS. More specifically we investigate the capability of neural speaker embeddings [16,17,19] to capture and model characteristics of speakers that were unseen during TTS model training. For this purpose, we extend an improved Tacotron system in [28] to a multi-speaker TTS system and conduct systematic analysis to answer the above question.…”