TristouNet: Triplet loss for speaker turn embedding

Abstract

TristouNet is a neural network architecture based on Long Short-Term Memory recurrent networks, meant to project speech sequences into a fixed-dimensional euclidean space. Thanks to the triplet loss paradigm used for training, the resulting sequence embeddings can be compared directly with the euclidean distance, for speaker comparison purposes. Experiments on short (between 500ms and 5s) speech turn comparison and speaker change detection show that TristouNet brings significant improvements over the current state-of-the-art techniques for both tasks.

A domain adaptation approach to improve speaker turn embedding using face representation

“…In this paper, we only consider the development set to compare with state-of-the-art methods. Specifically, we use similar settings for the "same/different" audio experiments than in [5]. The models learned with REPERE will be applied on ETAPE to benchmark their generalization ability.…”

“…We compare our speaker turn embedding with 3 approaches: Bayesian Information Criterion (BIC) [8], Gaussian divergence (Div.) [2], and the original TristouNet [5].…”

“…When these conditions are not valid, in particular the assumption of speaker turn duration, the quality of speaker diarization deteriorates [29], which was the case in TV series or movies [4,9] or in human-robot interactions where backchannels and short answers/utterances are very frequent. In this paper, we propose an approach to improve speaker turn representation by combining recent deep metric learning advances [5] with the adaptation of the face embedding domain.…”

“…Indeed recently, learning face embeddings has made significant achievements in all tasks, including recognition, verification, and clustering [24,30]. On the acoustic domain, we employ the speaker turn embedding trained with triplet loss (TristouNet) in [5], which achieved improvement on short utterances. By projecting both acoustic signals and face images into a common hypersphere, one can unify the two embedding spaces, thus enabling the knowledge to be shared across modalities.…”

Robust and Discriminative Speaker Embedding via Intra-Class Distance Variance Regularization

“…First, it can be extracted as the derivatives of the speaker recognition task by using the activation of the last layer before classification [4,6,7]. Second, it can be learned directly by optimizing the loss functions constraining the distances between same-speaker and different-speaker utterance pairs [3,8,9]. Among the distance-based losses, triplet loss has become more and more widely used in deep embedding networks [2,3,8].…”