Artificial neural network features for speaker diarization

Abstract

Speaker diarization finds contiguous speaker segments in an audio recording and clusters them by speaker identity, without any a-priori knowledge. Diarization is typically based on short-term spectral features such as Mel-frequency cepstral coefficients (MFCCs). Though these features carry average information about the vocal tract characteristics of a speaker, they are also susceptible to factors unrelated to the speaker identity. In this study, we propose an artificial neural network (ANN) architecture to learn a feature transform that is optimized for speaker diarization. We train a multi-hidden-layer ANN to judge whether two given speech segments came from the same or different speakers, using a shared transform of the input features that feeds into a bottleneck layer. We then use the bottleneck layer activations as features, either alone or in combination with baseline MFCC features in a multistream mode, for speaker diarization on test data. The resulting system is evaluated on various corpora of multi-party meetings. A combination of MFCC and ANN features gives up to 14% relative reduction in diarization error, demonstrating that these features are providing an additional independent source of knowledge.Index Terms-speaker diarization, artificial neural networks, discriminative feature extraction

Speaker Diarisation Using 2D Self-attentive Combination of Embeddings

“…The full training set which contains 135 meetings with 149 speakers was used which is further split into 90% for model training and 10% cross validation set for hyper-parameter tuning. For evaluation, instead of using the full dev and eval sets, we use the meetings recorded at IDIAP, Edinburgh and Brno which are the sets frequently used for evaluation of speaker diarisation [7,20], and which are more consistent with our observations on other datasets. The partition of the dataset is shown in Table 1.…”

Improved Large-Margin Softmax Loss for Speaker Diarisation

“…[1,2] Over the past years with the introduction of deep learning, diarisation systems have greatly improved in performance [3][4][5]. Going from the use of i-vectors to d-vectors, for representing a segment of speech for a single speaker, has been a contributing factor to progress [6,7]. The second factor has been the use of more sophisticated clustering processes, which are more appropriate for diarisation [8,9].…”

Capturing Suprasegmental Features of a Voice with RNNs for Improved Speaker Clustering

“…Recently, deep neural networks (DNN) have been successfully applied to various speaker recognition tasks [22,7,45,37,28,24], reaching and exceeding state of the art results of classic GMM- [35] or i-vector-based [8] systems. With few exceptions, systems based on convolutional neural network (CNN) architectures have been used on spectrograms for their unprecedented performance on visual recognition tasks [21].…”