Incremental Transfer Learning in Two-pass Information Bottleneck Based Speaker Diarization System for Meetings

Dawalatabad, Nauman; Madikeri, Srikanth; Sekhar, C. Chandra; Murthy, Hema A.

doi:10.1109/icassp.2019.8683114

Cited by 8 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is worth noting that although AMI is widely used for speaker diarisation, most studies either use different sets of meetings for testing, or audios recorded by independent headset microphones, such as (Bullock et al, 2020;Dawalatabad et al, 2019;Pal et al, 2020a), which makes such results not comparable to those presented in this paper. The official speech recognition data partition and MDM audios are used as a realistic setup for meeting transcription.…”

Section: Dataset Detailsmentioning

confidence: 83%

Combination of Deep Speaker Embeddings for Diarisation

Sun,

Zhang,

Woodland

2020

Preprint

View full text Add to dashboard Cite

Combining deep speaker embedding (window-level d-vector) extraction systems using 2D self-attentive, gated additive and bilinear pooling methods. The best-performing structure for combination is obtained by stacking a 2D self-attentive and a bilinear pooling structures.• A complete single pass neural network-based diarisation pipeline is introduced, which includes neural voice activity detection, neural change point detection, a deep speaker embedding extraction system and spectral clustering.• Experiments on both AMI and NIST RT05 evaluation sets showed that our proposed methods can produce state-ofthe-art results for the very challenging multi-speaker (with 4-10 speakers) meeting diarisation task.• We use the AMI dataset based on the official speech recognition partition with the audios recorded by multiple distance microphones (MDM) since it is a more realistic setup for meeting transcription than many different setups used by previous AMI based studies. This is the first paper to use this setup to the best of the authors' knowledge. Although this makes our results not directly comparable to those from the previous papers, we think our system still shows superior performance, since the results shown in Table 11 are the lowest diarisation error rates with the same training data, and the realistic setup we used can increase the difficulty of the task.• This paper is made based on our previous conference paper (Sun et al., 2019). Compared to our previous paper -Almost all of the combination structures are newly proposed and not included in (Sun et al., 2019), apart from the first type of 2D self-attentive method defined in Eqn. (4).-Sun et al. ( 2019) only performed the experiments on the AMI data with manual segmentation, while this paper explores to use both manual and automatic segmentation. The neural diarisation pipeline is newly introduced in this paper, although the neural VAD structure was proposed in (Wang et al., 2016).

show abstract

Section: Dataset Detailsmentioning

confidence: 83%

Combination of Deep Speaker Embeddings for Diarisation

Sun,

Zhang,

Woodland

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For finding the speaker clusters in a sequence of x-vectors, the variational Bayesian hidden Markov model (VBx) was investigated in [15,7]. For continuously learning speaker discriminative information, "Remember-Learn-Transfer" was proposed in [16]. Applying lexical and acoustic information for SD was investigated in [17].…”

Section: Related Workmentioning

confidence: 99%

Bertraffic: Bert-Based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications

Zuluaga-Gómez

Sarfjoo

Prasad

et al. 2023

2022 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Automatic speech recognition (ASR) allows transcribing the communications between air traffic controllers (ATCOs) and aircraft pilots. The transcriptions are used later to extract ATC named entities, e.g., aircraft callsigns. One common challenge is speech activity detection (SAD) and speaker diarization (SD). In the failure condition, two or more segments remain in the same recording, jeopardizing the overall performance. We propose a system that combines SAD and a BERT model to perform speaker change detection and speaker role detection (SRD) by chunking ASR transcripts, i.e., SD with a defined number of speakers together with SRD. The proposed model is evaluated on real-life public ATC databases. Our BERT SD model baseline reaches up to 10% and 20% token-based Jaccard error rate (JER) in public and private ATC databases. We also achieved relative improvements of 32% and 7.7% in JERs and SD error rate (DER), respectively, compared to VBx, a well-known SD system. 1

show abstract

“…An incremental transfer learning based approach was proposed recently by us, which uses the "Remember-Learn- Transfer" approach to continuously learn speaker discriminative information [34]. This reduces the RTF of TPIB (with MFNN) system [8] by 33% (relative) but with minor degradation in the speaker error rate.…”

Section: Approaches To Speaker Diarizationmentioning

confidence: 99%

“…The list of meeting IDs from AMI datasets recorded at Idiap (IS) and Edinburgh (ES) is given in Table I. The datasets are also used in [8] and [34] for evaluation. The number of speakers in the NIST dataset varies from 3 to 10.…”

Section: A Datasets Features and Evaluation Measurementioning

confidence: 99%

Novel Architectures for Unsupervised Information Bottleneck based Speaker Diarization of Meetings

Dawalatabad,

Madikeri,

Sekhar

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Speaker diarization is an important problem that is topical, and is especially useful as a preprocessor for conversational speech related applications. The objective of this paper is two-fold: (i) segment initialization by uniformly distributing speaker information across the initial segments, and (ii) incorporating speaker discriminative features within the unsupervised diarization framework. In the first part of the work, a varying length segment initialization technique for Information Bottleneck (IB) based speaker diarization system using phoneme rate as the side information is proposed. This initialization distributes speaker information uniformly across the segments and provides a better starting point for IB based clustering. In the second part of the work, we present a Two-Pass Information Bottleneck (TPIB) based speaker diarization system that incorporates speaker discriminative features during the process of diarization. The TPIB based speaker diarization system has shown improvement over the baseline IB based system. During the first pass of the TPIB system, a coarse segmentation is performed using IB based clustering. The alignments obtained are used to generate speaker discriminative features using a shallow feed-forward neural network and linear discriminant analysis. The discriminative features obtained are used in the second pass to obtain the final speaker boundaries. In the final part of the paper, variable segment initialization is combined with the TPIB framework. This leverages the advantages of better segment initialization and speaker discriminative features that results in an additional improvement in performance. An evaluation on standard meeting datasets shows that a significant absolute improvement of 3.9% and 4.7% is obtained on the NIST and AMI datasets, respectively.

show abstract

Incremental Transfer Learning in Two-pass Information Bottleneck Based Speaker Diarization System for Meetings

Cited by 8 publications

References 18 publications

Combination of Deep Speaker Embeddings for Diarisation

Combination of Deep Speaker Embeddings for Diarisation

Bertraffic: Bert-Based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications

Novel Architectures for Unsupervised Information Bottleneck based Speaker Diarization of Meetings

Contact Info

Product

Resources

About