Improved overlap speech diarization of meeting recordings using long-term conversational features

Yella, Sree Harsha; Bourlard, Hervé

doi:10.1109/icassp.2013.6639171

Cited by 19 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various methods have been proposed to solve the problem of overlapping speaker segmentation including hidden markov model (HMM)-based methods that use Mel-frequency cepstral coefficients (MFCCs), linear predictive coding (LPC) and root mean square (RMS) energy features [12]. Methods that use long-term conversational features [13] have also been put forward along with multimodal techniques that use multiple microphone and camera systems [14]. More recently, deep learning approaches have become increasingly prevalent [15]- [18] which often require large amounts of labelled training data.…”

Section: Introductionmentioning

confidence: 99%

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Hogg

Evers

Moore

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

This paper demonstrates how the harmonic structure of voiced speech can be exploited to segment multiple overlapping speakers in a speaker diarization task. We explore how a change in the speaker can be inferred from a change in pitch. We show that voiced harmonics can be useful in detecting when more than one speaker is talking, such as during overlapping speaker activity. A novel system is proposed to track multiple harmonics simultaneously, allowing for the determination of onsets and end-points of a speaker's utterance in the presence of an additional active speaker. This system is bench-marked against a segmentation system from the literature that employs a bidirectional long short term memory network (BLSTM) approach and requires training. Experimental results highlight that the proposed approach outperforms the BLSTM baseline approach by 12.9% in terms of HIT rate for speaker segmentation. We also show that the estimated pitch tracks of our system can be used as features to the BLSTM to achieve further improvements of 1.21% in terms of coverage and 2.45% in terms of purity.

show abstract

Section: Introductionmentioning

confidence: 99%

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Hogg

Evers

Moore

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Existing approaches to the segmentation of overlapping speech include [10] where a HMM-based method, that used mel-frequency cepstral coefficients (MFCCs), linear predictive coding (LPC) and root mean square (RMS) energy features, was proposed. It has also been shown in [11] that the segmentation performance can be improved if long-term conversational features are utilised. Other methods include multimodal techniques that use multiple microphone and camera systems [12].…”

Section: Introductionmentioning

confidence: 99%

Multiple Hypothesis Tracking for Overlapping Speaker Segmentation

Hogg

Evers

Naylor

2019

2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)

View full text Add to dashboard Cite

Speaker segmentation is an essential part of any diarization system. Applications of diarization include tasks such as speaker indexing, improving automatic speech recognition (ASR) performance and making single speaker-based algorithms available for use in multi-speaker environments. This paper proposes a multiple hypothesis tracking (MHT) method that exploits the harmonic structure associated with the pitch in voiced speech in order to segment the onsets and end-points of speech from multiple, overlapping speakers. The proposed method is evaluated against a segmentation system from the literature that uses a spectral representation and is based on employing bidirectional long short term memory networks (BLSTM). The proposed method is shown to achieve comparable performance for segmenting overlapping speakers only using the pitch harmonic information in the MHT framework.

show abstract

“…An approach presented in [15] uses the output of a voice activity detection system and the silence distribution to detect overlap. This work was extended by exploiting long-term conversational features for overlap detection [16]. Neural networks could improve overlap detection by analysing the context.…”

Section: Introductionmentioning

confidence: 99%

Detecting overlapping speech with long short-term memory recurrent neural networks

et al. 2013

View full text Add to dashboard Cite

Detecting segments of overlapping speech (when two or more speakers are active at the same time) is a challenging problem. Previously, mostly HMM-based systems have been used for overlap detection, employing various different audio features. In this work, we propose a novel overlap detection system using Long Short-Term Memory (LSTM) recurrent neural networks. LSTMs are used to generate framewise overlap predictions which are applied for overlap detection. Furthermore, a tandem HMM-LSTM system is obtained by adding LSTM predictions to the HMM feature set. Experiments with the AMI corpus show that overlap detection performance of LSTMs is comparable to HMMs. The combination of HMMs and LSTMs improves overlap detection by achieving higher recall.

show abstract

Improved overlap speech diarization of meeting recordings using long-term conversational features

Cited by 19 publications

References 9 publications

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

Multiple Hypothesis Tracking for Overlapping Speaker Segmentation

Detecting overlapping speech with long short-term memory recurrent neural networks

Contact Info

Product

Resources

About