Active correction for speaker diarization with human in the loop

Abstract

State of the art diarization systems now achieve decent performance but those performances are often not good enough to deploy them without any human supervision. In this paper we propose a framework that solicits a human in the loop to correct the clustering by answering simple questions. After defining the nature of the questions, we propose an algorithm to list those questions and two stopping criteria that are necessary to limit the work load on the human in the loop. Experiments performed on the ALLIES dataset show that a limited interaction with a human expert can lead to considerable improvement of up to 36.5% relative diarization error rate (DER) compared to a strong baseline.

Active Correction for Incremental Speaker Diarization of a Collection with Human in the Loop

“…Max/Min selects the couple of segments, one from each branch, with the lowest (max) or highest (min) similarity in terms of PLDA score (distance). The results of our study [10] showed that Max/Min criterion are not competitive and we thus only focus on Longest and Cluster center in this work.…”

“…Final performance assessment of a human assisted system must take into account the cost of human interaction in order to evaluate the quality of the interaction process. Penalized DER (DER pen ), a metric introduced in [10], is used to merge the information about the final performance (after human interaction) with the cost of the interaction required to reach this result. This metric adds a constant amount of error time, called penalized time (t pen ), to the diarization error time, for each question asked to the human expert.…”

“…The system must select the two most representative or informative samples. In a previous work [10], we investigated 4 sample selection methods:…”

“…Amongst the different modes of human assisted learning, our work focuses on active learning where the automatic system, while processing an incoming sequence of audio recordings, is allowed to ask simple questions to the human expert [9]. In a previous work, we proposed a human-assisted correction process for within-show speaker diarization [10].…”

Multimodal Clustering with Role Induced Constraints for Speaker Diarization

“…Constraints can be combined with several clustering algorithms, such as k-means [20] or HAC [21]. In this work we use a constrained spectral clustering approach, where constraints are integrated via the exhaustive and efficient constraint propagation (E 2 CP) algorithm [22], which was recently applied in diarization settings [19].…”