Detection of Overlapping Speech for the Purposes of Speaker Diarization

Kunešová, Marie; Hrúz, Marek; Zajíc, Zbyněk; Radová, Vlasta

doi:10.1007/978-3-030-26061-3_26

Cited by 25 publications

(20 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[7] defend the use of artificially mixed data for training in order to combat the imbalance of overlapped and monospeaker regions. Most recently, [8] report CNN-based overlap detection accuracy and evaluate the resulting potential change in diarization error rate (DER), but assume access to perfect two-label assignment.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection

Bullock

Bredin

García-Perera

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

We address the problem of effectively handling overlapping speech in a diarization system. First, we detail a neural Long Short-Term Memory-based architecture for overlap detection. Secondly, detected overlap regions are exploited in conjunction with a frame-level speaker posterior matrix to make two-speaker assignments for overlapped frames in the resegmentation step. The overlap detection module achieves state-of-the-art performance on the AMI, DIHARD, and ETAPE corpora. We apply overlap-aware resegmentation on AMI, resulting in a 20% relative DER reduction over the baseline system. While this approach is by no means an endall solution to overlap-aware diarization, it reveals promising directions for handling overlap.

show abstract

Section: Introductionmentioning

confidence: 99%

“…https://github.com/jsalt2019-diadet/jsalt2019-diadet3 Thanks to Claude Barras for providing the overlapped speech detection output corresponding to system L 1 inTable 2of[20], and Marie Kunešová for providing the overlapped speech detection output corresponding to system "AMI test (all subsets) + dereverberation" inTable 2of[8].…”

mentioning

confidence: 99%

Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection

Bullock

Bredin

García-Perera

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Among all pyannote.audio alternatives, it is the most similar: written in Python, it provides most of the afore-This research was partly funded by the French National Research Agency (ANR) through the ODESSA (ANR-15-CE39-0010) and PLUM-COT (ANR-16-CE92-0025) projects. We would like to thank Claude Barras for providing the overlapped speech detection output corresponding to system L 1 in Table 2 of [1], Neville Ryant for the speaker diarization output of the winning submission to DIHARD 2019 [2,3], Marie Kunešová for the overlapped speech detection output corresponding to system "AMI test (all subsets) + dereverberation" in Table 2 of [4], and Sylvain Meignier for the speaker diarization output of [5] on ETAPE dataset. mentioned blocks, and goes all the way down to the actual evaluation of the system.…”

Section: Introductionmentioning

confidence: 99%

Pyannote.Audio: Neural Building Blocks for Speaker Diarization

Bredin¹,

Yin²,

Coria³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

206

128

View full text Add to dashboard Cite

We introduce pyannote.audio, an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it provides a set of trainable end-to-end neural building blocks that can be combined and jointly optimized to build speaker diarization pipelines. pyannote.audio also comes with pre-trained models covering a wide range of domains for voice activity detection, speaker change detection, overlapped speech detection, and speaker embedding -reaching state-of-the-art performance for most of them.

show abstract

“…The segmentation task is often complicated by the presence of overlapping speech where multiple speakers are active at the same time [7], [8]. Overlapping speech commonly occurs in conversational speech due to interruptions and backchannel vocalisations [9]. Environmental factors, such as reverberation [10] and noise [11], also render the task of accurate segmentation difficult to achieve.…”

Section: Introductionmentioning

confidence: 99%

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Hogg

Evers

Moore

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

This paper demonstrates how the harmonic structure of voiced speech can be exploited to segment multiple overlapping speakers in a speaker diarization task. We explore how a change in the speaker can be inferred from a change in pitch. We show that voiced harmonics can be useful in detecting when more than one speaker is talking, such as during overlapping speaker activity. A novel system is proposed to track multiple harmonics simultaneously, allowing for the determination of onsets and end-points of a speaker's utterance in the presence of an additional active speaker. This system is bench-marked against a segmentation system from the literature that employs a bidirectional long short term memory network (BLSTM) approach and requires training. Experimental results highlight that the proposed approach outperforms the BLSTM baseline approach by 12.9% in terms of HIT rate for speaker segmentation. We also show that the estimated pitch tracks of our system can be used as features to the BLSTM to achieve further improvements of 1.21% in terms of coverage and 2.45% in terms of purity.

show abstract

Detection of Overlapping Speech for the Purposes of Speaker Diarization

Cited by 25 publications

References 15 publications

Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection

Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection

Pyannote.Audio: Neural Building Blocks for Speaker Diarization

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Contact Info

Product

Resources

About