Scenario-Dependent Speaker Diarization for DIHARD-III Challenge

Wang, Yu-Xuan; Du, Jun; He, Maokui; Niu, Shutong; Sun, Lei; Lee, Chin‐Hui

doi:10.21437/interspeech.2021-516

Cited by 17 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The deployment of i-vectors in TS-VAD, while successful to some extent, showed limitations when applied to multiscenario data [58]. These findings pave the way for the exploration of x-vectors as an alternative, but a simple swap of i-vectors for x-vectors did not yield an immediate boost in performance [19].…”

Section: Target Speaker Voice Activity Detectionmentioning

confidence: 97%

Robust End-to-end Speaker Diarization with Generic Neural Clustering

Yang¹,

Wang²

2022

Interspeech 2022

View full text Add to dashboard Cite

This paper proposes an online target speaker voice activity detection system for speaker diarization tasks, which does not require a priori knowledge from the clustering-based diarization system to obtain the target speaker embeddings. By adapting the conventional target speaker voice activity detection for real-time operation, this framework can identify speaker activities using self-generated embeddings, resulting in consistent performance without permutation inconsistencies in the inference phase. During the inference process, we employ a front-end model to extract the frame-level speaker embeddings for each coming block of a signal. Next, we predict the detection state of each speaker based on these frame-level speaker embeddings and the previously estimated target speaker embedding. Then, the target speaker embeddings are updated by aggregating these framelevel speaker embeddings according to the predictions in the current block. Our model predicts the results for each block and updates the target speakers' embeddings until reaching the end of the signal. Experimental results show that the proposed method outperforms the offline clustering-based diarization system on the DIHARD III and AliMeeting datasets. The proposed method is further extended to multi-channel data, which achieves similar performance with the state-of-the-art offline diarization systems.

show abstract

Section: Target Speaker Voice Activity Detectionmentioning

confidence: 97%

Robust End-to-end Speaker Diarization with Generic Neural Clustering

Yang¹,

Wang²

2022

Interspeech 2022

View full text Add to dashboard Cite

show abstract

“…The voice activity is recognized by the characteristic's speaker signal, as extracted by utilizing the Eq. (11).…”

Section: Voice Activity Detectionmentioning

confidence: 99%

“…The major component of this differentiation is clustering, in which the segmentation involves the bottom-up, global optimization, neural network clustering, as well as up-down approach. The existing approaches for clustering contain Deep Neural Network (DNN), spectral clustering, and bottleneck-based methods [11]. The existing methods of the SD have traversed by clustering free methods or end-to-end approaches by multiple speakers' discussion [12].…”

Section: Introductionmentioning

confidence: 99%

A Hybrid DMFCC-LPC Based Feature Extraction with DCNN Clustering for Speaker Diarization

2024

IJIES

View full text Add to dashboard Cite

Speaker Diarization (SD) or speaker indexing is a procedure for automatically partitioning a conversation by the number of speakers into homogeneous segments. The trustworthy diarization method accurately estimates the variable length assertion, and it involves major steps such as speech detection, speaker merges, and speaker change. The major problem with the SD method is enhancing the readability of speech transcription. In this study, a hybrid method of feature extraction based on the Dynamic Mel Frequency Cepstral Coefficient (DMFCC) and Linear Prediction Coding (LPC) is proposed for SD. The Voice Activity Detection (VAD) method is utilized to detect the presence or absence of a speaker in the audio lecture, which is followed by speaker segmentation utilizing the extracted features. The Deep Convolutional Neural Network (DCNN) is utilized to determine the feature vector and cluster the speaker from the audio lecture. The results show that the proposed DMFCC-LPC delivers a robust performance on metrics such as accuracy, Diarization Error Rate (DER) and False Positive Rate (FPR) of 0.967, 0.31 and 0.119 on CALLHOME dataset, in contrast to the Speaker Siarization System using HXLP-DCNN with Sailfish Optimization Algorithm (SDS-HXLP-DCNN-SOA), Feature-Level fusion, and Self-supervised clustering with Path Integral Clustering (SSC-PIC).

show abstract

“…Later, LSTM [63] and Transformer [20] modules are implemented along the speaker dimension of models to handle a variable number of speakers. On the other hand, the i-vectors used in TS-VAD are relatively domain-dependent, restricting the system performance on multi-scenario dataset [64]. This finding paves the way for exploring more discriminative speaker embeddings like x-vectors as an alternative.…”

Section: Target-speaker Voice Activity Detectionmentioning

confidence: 99%

Online Target Speaker Voice Activity Detection for Speaker Diarization

Wang¹,

Li²,

Lin³

2022

Interspeech 2022

View full text Add to dashboard Cite

Audio-visual learning has demonstrated promising results in many classical speech tasks (e.g., speech separation, automatic speech recognition, wake-word spotting). We believe that introducing visual modality will also benefit speaker diarization. To date, Target-Speaker Voice Activity Detection (TS-VAD) plays an important role in highly accurate speaker diarization. However, previous TS-VAD models take audio features and utilize the speaker's acoustic footprint to distinguish his or her personal speech activities, which is easily affected by overlapped speech in multi-speaker scenarios. Although visual information naturally tolerates overlapped speech, it suffers from spatial occlusion, low resolution, etc. The potential modality-missing problem blocks TS-VAD towards an audio-visual approach.This paper proposes a novel Multi-Input Multi-Output Target-Speaker Voice Activity Detection (MIMO-TSVAD) framework for speaker diarization. The proposed method can take audio-visual input and leverage the speaker's acoustic footprint or lip track to flexibly conduct audio-based, video-based, and audio-visual speaker diarization in a unified sequence-to-sequence framework. Experimental results show that the MIMO-TSVAD framework demonstrates state-of-the-art performance on the VoxConverse, DIHARD-III, and MISP 2022 datasets under corresponding evaluation metrics, obtaining the Diarization Error Rates (DERs) of 4.18%, 10.10%, and 8.15%, respectively. In addition, it can perform robustly in heavy lip-missing scenarios.

show abstract

Scenario-Dependent Speaker Diarization for DIHARD-III Challenge

Cited by 17 publications

References 17 publications

Robust End-to-end Speaker Diarization with Generic Neural Clustering

Robust End-to-end Speaker Diarization with Generic Neural Clustering

A Hybrid DMFCC-LPC Based Feature Extraction with DCNN Clustering for Speaker Diarization

Online Target Speaker Voice Activity Detection for Speaker Diarization

Contact Info

Product

Resources

About