Low-latency speaker spotting with online diarization and detection

Abstract

This paper introduces a new task termed low-latency speaker spotting (LLSS). Related to security and intelligence applications, the task involves the detection, as soon as possible, of known speakers within multi-speaker audio streams. The paper describes differences to the established fields of speaker diarization and automatic speaker verification and proposes a new protocol and metrics to support exploration of LLSS. These can be used together with an existing, publicly available database to assess the performance of LLSS solutions also proposed in the paper. They combine online diarization and speaker detection systems. Diarization systems include a naive, over-segmentation approach and fully-fledged online diarization using segmental i-vectors. Speaker detection is performed using Gaussian mixture models, i-vectors or neural speaker embeddings. Metrics reflect different approaches to characterise latency in addition to detection performance. The relative performance of each solution is dependent on latency. When higher latency is admissible, i-vector solutions perform well; embeddings excel when latency must be kept to a minimum. With a need to improve the reliability of online diarization and detection, the proposed LLSS framework provides a vehicle to fuel future research in both areas. In this respect, we embrace a reproducible research policy; results can be readily reproduced using publicly available resources and open source codes.

Speaker Recognition Based on Deep Learning: An Overview

“…Online speaker diarization, which outputs the diarization result right after the audio segment arrives, is not an easy task, since that future information is unavailable when analyzing the current segment [215]. In history, a number of online speaker diarization and speaker tracking solutions have been reported [216,217]. Here we focus on the deep learning based ones, which can be categorized to stage-wise online diarization and end-to-end online diarization methods.…”

Probabilistic Back-ends for Online Speaker Recognition and Clustering

“…guests) leading to an open-set identification task. Another related task is low-latency speaker spotting [3], where a previously registered target speaker has to be detected in an audio stream.…”

Online End-To-End Neural Diarization with Speaker-Tracing Buffer

“…State-of-art speaker diarization systems mostly concentrate on integrating several components: voice activity detection, speaker change detection, feature representation, and clustering [11,12]. Current research focuses primarily on the speaker model or speaker embeddings, such as Gaussian mixture models (GMM) [8,13], i-vector [14][15][16], d-vector [17,18], and x-vector [19,20], and on a better clustering method such as agglomerative hierarchical clustering or spectral clustering [19,[21][22][23]. The issue with these methods is that they cannot directly minimize the diarization error because they are based on an unsupervised algorithm.…”