Speaker change detection using features through a neural network speaker classifier

Abstract

The mechanism proposed here is for real-time speaker change detection in conversations, which firstly trains a neural network text-independent speaker classifier using indomain speaker data. Through the network, features of conversational speech from out-of-domain speakers are then converted into likelihood vectors, i.e. similarity scores comparing to the in-domain speakers. These transformed features demonstrate very distinctive patterns, which facilitates differentiating speakers and enable speaker change detection with some straight-forward distance metrics. The speaker classifier and the speaker change detector are trained/tested using speech of the first 200 (indomain) and the remaining 126 (out-of-domain) male speakers in TIMIT respectively. For the speaker classification, 100% accuracy at a 200 speaker size is achieved on any testing file, given the speech duration is at least 0.97 seconds. For the speaker change detection using speaker classification outputs, performance based on 0.5, 1, and 2 seconds of inspection intervals were evaluated in terms of error rate and F1 score, using synthesized data by concatenating speech from various speakers. It captures close to 97% of the changes by comparing the current second of speech with the previous second, which is very competitive among literature using other methods.

An Approach to Online Speaker Change Point Detection Using DNNs and WFSTs

“…The use of i-vectors was also investigated in [32]. Features extracted from DNN were explored in [33]. Finally, the authors in [34] studied in detail the influence of the online environment on various SCP detection approaches to diarization systems.…”

Sequence-Level Speaker Change Detection With Difference-Based Continuous Integrate-and-Fire

“…SCD is known as an important part of speaker diarization [6], [7]. It was previously modeled with distance-based methods [3], [8], [9], which segment audio with a sliding window, and the distance of speaker embedding is used to decide whether a speaker change happens between the adjacent segments. Since the pitch varies saliently with speaker changes, some pitch-based methods detect speaker changes with the change in pitch [10]- [12].…”

Token-level Speaker Change Detection Using Speaker Difference and Speech Content via Continuous Integrate-and-fire

“…The distance-based methods [1][2][3][4][5] are studied earliest, which calculate the distance between features in the adjacent windows, and once the distance exceeds the threshold, a speaker change is detected. In addition, the model-based methods [6][7][8][9] segment the input audio into fix-length segments that are assumed to contain only one speaker, and models are trained to extract speaker embeddings for each segment. Then the distance between adjacent speaker embeddings is used to decide whether speaker change happens.…”