Development of Supervised Speaker Diarization System Based on the PyAnnote Audio Processing Library

Abstract

Diarization is an important task when work with audiodata is executed, as it provides a solution to the problem related to the need of dividing one analyzed call recording into several speech recordings, each of which belongs to one speaker. Diarization systems segment audio recordings by defining the time boundaries of utterances, and typically use unsupervised methods to group utterances belonging to individual speakers, but do not answer the question “who is speaking?” On the other hand, there are biometric systems that identify individuals on the basis of their voices, but such systems are designed with the prerequisite that only one speaker is present in the analyzed audio recording. However, some applications involve the need to identify multiple speakers that interact freely in an audio recording. This paper proposes two architectures of speaker identification systems based on a combination of diarization and identification methods, which operate on the basis of segment-level or group-level classification. The open-source PyAnnote framework was used to develop the system. The performance of the speaker identification system was verified through the application of the AMI Corpus open-source audio database, which contains 100 h of annotated and transcribed audio and video data. The research method consisted of four experiments to select the best-performing supervised diarization algorithms on the basis of PyAnnote. The first experiment was designed to investigate how the selection of the distance function between vector embedding affects the reliability of identification of a speaker’s utterance in a segment-level classification architecture. The second experiment examines the architecture of cluster-centroid (group-level) classification, i.e., the selection of the best clustering and classification methods. The third experiment investigates the impact of different segmentation algorithms on the accuracy of identifying speaker utterances, and the fourth examines embedding window sizes. Experimental results demonstrated that the group-level approach offered better identification results were compared to the segment-level approach, and the latter had the advantage of real-time processing.

A Hybrid DMFCC-LPC Based Feature Extraction with DCNN Clustering for Speaker Diarization

“…Khoma [17] developed a multi-architecture of the speaker recognition systems based on the integration of identification and diarization approach. This approach was segmented according to segment-level or group-level classification, aside from an opensource PyAnnote module utilized to build the system.…”

Comparison of Modern Deep Learning Models for Speaker Verification

“…Studies [3,29] present the results of utilizing the Pyannote and ECAPA models in diarization tasks rather than verification tasks.…”

“…Traditionally, speaker diarization involves both segmentation (detecting speaker changes in an audio recording) and clustering (grouping speech segments corresponding to specific speakers based on voice characteristics). Unlike speaker verification and identification, speaker diarization can be approached using unsupervised learning [3].…”

Implementation of a Hybrid Intelligence System Enabling the Effectiveness Assessment of Interaction Channels Use in HMI

“…This collaboration is expected to take place using natural communication, instead of specialized interfaces or protocols, shifting the focus towards human-oriented, context-aware, adaptive systems [5]. With the progress of Artificial Intelligence (AI), as well as NLP (Natural Language Processing), which is clearly seen in modern conversational {systems [6,7], the man-machine collaboration becomes an urgent and promising trend in modern industry. Nevertheless, despite recent significant advancements [8][9][10][11] in the field of Human-Computer Interaction (HCI) (namely: the Amazon Echo available since 2015 [12][13][14], Microsoft Cortana massively expanding in 2015 to numerous platforms [15], Google Speech announced in 2016, upgraded in 2018 [16] and 2022 [17], Google Assistant announced in 2016 [18,19], Google Nest introduced as Google Home also in 2016 [20][21][22][23][24], and the Apple Siri updated in 2017 [25,26]) and undoubtful progress in Artificial Intelligence and Machine Learning [27][28][29][30], conversational AI systems usually still tend to disappoint rather than to amaze their interlocutors (and there are some specific reasons for that, already diagnosed and described [31]).…”