Speaker Diarization Using Convolutional Neural Network for Statistics Accumulation Refinement

Abstract

The aim of this paper is to investigate the benefit of information from a speaker change detection system based on Convolutional Neural Network (CNN) when applied to the process of accumulation of statistics for an i-vector generation. The investigation is carried out on the problem of diarization. In our system, the output of the CNN is a probability value of a speaker change in a conversation for a given time segment. According to this probability, we cut the conversation into short segments that are then represented by the i-vector (to describe a speaker in it). We propose a technique to utilize the information from the CNN for the weighting of the acoustic data in a segment to refine the statistics accumulation process. This technique enables us to represent the speaker better in the final i-vector. The experiments on the English part of the CallHome corpus show that our proposed refinement of the statistics accumulation is beneficial with the relative improvement of Diarization Error Rate almost by 16 % when compared to the speaker diarization system without statistics refinement.

Fully Supervised Speaker Diarization

“…For the embedding extraction module, recent work [2,3,7] has shown that the diarization performance can be significantly improved by replacing i-vectors [5] with neural network embeddings, a.k.a. d-vectors [6,8].…”

Latent class model with application to speaker diarization

“…The number of speakers is assumed to be known in advance. Figure. 8 shows the DER of 'TL 7' varies with k of soft prior (22). According to the variation trend, we choose k = 10 in our experiment.…”

Experiments with Segmentation in an Online Speaker Diarization System

“…For comparison, we also show the results of the offline system (adapted from our previous works [8,18]). Table 1.…”

“…Decision threshold for the online approach was θ = 0.6. Offline results (except oracle) were adapted from [8,18]. The table shows that in the offline scenario, the naïve fixed length segmentation produces reasonable results (likely due to resegmentation [16]), although it is surpassed by the CNN-based approach.…”

“…Additionally, we also utilize the information about the change P from the CNN for weighting of the acoustic data in a segment, in order to refine the statistics accumulation process used for the subsequent i-vector generation [18].…”