TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge

Pang, Bowen; Zhao, Huan; Zhang, Gaosheng; Yang, Xiaoyue; Sun, Yang; Zhang, Li; Wang, Qing; Xie, Lei

doi:10.1109/iscslp57327.2022.10037846

Cited by 5 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system developed outputs the ensemble results of the four modules: self-attentive-based VAD, uniform segmentation, ECAPA-TDNN-based embedding extractor, and spectral clustering. This is the same challenge as in paper [12] above. stands for conversational short-phrase speaker diarization dataset and has three features.…”

Section: Fig -2mentioning

confidence: 87%

“…Table -2 It is evident that CDER falls with increasing duration. Also, the paper [12] shows that SC, TS-VAD and EEND are explored and then DOVER-LAP was applied to fuse the RTTM outputs inferred from the above three systems to get the result. Paper [12] shows that spectral clustering-based speaker diarization on metric CDER performs best with 12.0% and 9.5% on the dev set and test set respectively and hence is competitive with the new CDER metric.…”

Section: Fig -2mentioning

confidence: 99%

“…Fig -4: The influence of different sub-segment duration to DER (%) and CDER (%)It is evident that CDER falls with increasing duration. Also, the paper[12] shows that SC, TS-VAD and EEND are explored and then DOVER-LAP was applied to fuse the RTTM outputs inferred from the above three systems to get the result. Paper[12] shows that spectral clustering-based speaker diarization on metric CDER performs best with 12.0% and…”

mentioning

confidence: 99%

See 2 more Smart Citations

Speaker Diarization: A Review

Kumar¹

2023

IJSREM

View full text Add to dashboard Cite

Speaker diarization is the task of determining "who spoke when?" in an audio or video recording that contains an unknown amount of speech and an unknown number of speakers. It is a challenging task due to the variability of human speech, the presence of overlapping speech, and the lack of prior information about the speakers. It is the process of labeling a speech signal with labels corresponding to the identity of speakers. It is a crucial task in audio signal processing and speech analysis. A recent review of speaker diarization research since 2018 can be found in this paper which discusses the historical development of speaker diarization technology and recent advancements in neural speaker diarization approaches. Key Words: speaker diarization, speaker clustering, speaker embeddings

show abstract

Section: Fig -2mentioning

confidence: 87%

Section: Fig -2mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Speaker Diarization: A Review

Kumar¹

2023

IJSREM

View full text Add to dashboard Cite

show abstract

“…Using visual information as the complementary modality to improve diarization systems becomes a promising direction. Existing works mainly depend on constructing cross-modal synergy [25]- [28], clustering on audio-visual pairs [29], [30], or end-to-end audio-visual diarization [31], [32], which are basically derived from the previous audio-only methods. Motivated by the highly accurate performance in TS-VAD studies, a question arises if it is feasible to investigate this framework in an audio-visual manner.…”

Section: Introductionmentioning

confidence: 99%

Online Target Speaker Voice Activity Detection for Speaker Diarization

Wang¹,

Li²,

Lin³

2022

Interspeech 2022

View full text Add to dashboard Cite

Audio-visual learning has demonstrated promising results in many classical speech tasks (e.g., speech separation, automatic speech recognition, wake-word spotting). We believe that introducing visual modality will also benefit speaker diarization. To date, Target-Speaker Voice Activity Detection (TS-VAD) plays an important role in highly accurate speaker diarization. However, previous TS-VAD models take audio features and utilize the speaker's acoustic footprint to distinguish his or her personal speech activities, which is easily affected by overlapped speech in multi-speaker scenarios. Although visual information naturally tolerates overlapped speech, it suffers from spatial occlusion, low resolution, etc. The potential modality-missing problem blocks TS-VAD towards an audio-visual approach.This paper proposes a novel Multi-Input Multi-Output Target-Speaker Voice Activity Detection (MIMO-TSVAD) framework for speaker diarization. The proposed method can take audio-visual input and leverage the speaker's acoustic footprint or lip track to flexibly conduct audio-based, video-based, and audio-visual speaker diarization in a unified sequence-to-sequence framework. Experimental results show that the MIMO-TSVAD framework demonstrates state-of-the-art performance on the VoxConverse, DIHARD-III, and MISP 2022 datasets under corresponding evaluation metrics, obtaining the Diarization Error Rates (DERs) of 4.18%, 10.10%, and 8.15%, respectively. In addition, it can perform robustly in heavy lip-missing scenarios.

show abstract