A study of speaker clustering for speaker attribution in large telephone conversation datasets

Ghaemmaghami, Houman; Sridharan, Sridha; Leeuwen, David A. van

doi:10.1016/j.csl.2016.03.005

Cited by 11 publications

(13 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarity scores for all possible speaker pairs (i.e, the reference speakers with known speaker identities and the speakers with pseudo labels assigned by the SD system) are calculated. Based on these speaker similarity scores represented in the form of a distance matrix, speaker linking is performed by applying complete-linkage clustering as described in [39]. A mild level of linking is performed, as aggressive merging of the speaker labels conceivably has an inverse effect on the ASR performance, which has been also observed in the pilot experiments.…”

Section: Speaker Linkingmentioning

confidence: 95%

Semi-supervised acoustic model training for speech with code-switching

Yılmaz

McLaren

Heuvel

et al. 2018

Speech Communication

View full text Add to dashboard Cite

In the FAME! project, we aim to develop an automatic speech recognition (ASR) system for Frisian-Dutch code-switching (CS) speech extracted from the archives of a local broadcaster with the ultimate goal of building a spoken document retrieval system. Unlike Dutch, Frisian is a low-resourced language with a very limited amount of manually annotated speech data. In this paper, we describe several automatic annotation approaches to enable using of a large amount of raw bilingual broadcast data for acoustic model training in a semi-supervised setting. Previously, it has been shown that the best-performing ASR system is obtained by two-stage multilingual deep neural network (DNN) training using 11 hours of manually annotated CS speech (reference) data together with speech data from other high-resourced languages. We compare the quality of transcriptions provided by this bilingual ASR system with several other approaches that use a language recognition system for assigning language labels to raw speech segments at the front-end and using monolingual ASR resources for transcription. We further investigate automatic annotation of the speakers appearing in the raw broadcast data by first labeling with (pseudo) speaker tags using a speaker diarization system and then linking to the known speakers appearing in the reference data using a speaker recognition system. These speaker labels are essential for speaker-adaptive training in the proposed setting. We train acoustic models using the manually and automatically annotated data and run recognition experiments on the development and test data of the FAME! speech corpus to quantify the quality of the automatic annotations. The ASR and CS detection results demonstrate the potential of using automatic language and speaker tagging in semi-supervised bilingual acoustic model training.

show abstract

Section: Speaker Linkingmentioning

confidence: 95%

Semi-supervised acoustic model training for speech with code-switching

Yılmaz

McLaren

Heuvel

et al. 2018

Speech Communication

View full text Add to dashboard Cite

show abstract

“…A similarity matrix for all possible speaker pairs are calculated using a PLDA model [23] to find the cross-tape speaker similarities. Based on these speaker similarity scores, speaker linking is performed by applying completelinkage clustering as described in [7]. Varying levels of speaker linking can be performed by manipulating the clustering threshold results in different levels of speaker and cluster impurities.…”

Section: Second Stage: Speaker Linking and Identificationmentioning

confidence: 99%

Large-Scale Speaker Diarization of Radio Broadcast Archives

Yılmaz¹,

Derinel²,

Zhou³

et al. 2019

Interspeech 2019

View full text Add to dashboard Cite

This paper describes our initial efforts to build a large-scale speaker diarization (SD) and identification system on a recently digitized radio broadcast archive from the Netherlands which has more than 6500 audio tapes with 3000 hours of Frisian-Dutch speech recorded between 1950-2016. The employed large-scale diarization scheme involves two stages: (1) tapelevel speaker diarization providing pseudo-speaker identities and (2) speaker linking to relate pseudo-speakers appearing in multiple tapes. Having access to the speaker models of several frequently appearing speakers from the previously collected FAME! speech corpus, we further perform speaker identification by linking these known speakers to the pseudo-speakers identified at the first stage. In this work, we present a recently created longitudinal and multilingual SD corpus designed for large-scale SD research and evaluate the performance of a new speaker linking system using x-vectors with PLDA to quantify cross-tape speaker similarity on this corpus. The performance of this speaker linking system is evaluated on a small subset of the archive which is manually annotated with speaker information. The speaker linking performance reported on this subset (53 hours) and the whole archive (3000 hours) is compared to quantify the impact of scaling up in the amount of speech data.

show abstract

“…We have considered the conventional bottom-up AHC clustering system with the options of single and average linkages. We did not consider the model retraining approach because it is costly in terms of computations as compared to the linkage approaches to clustering [14]. The system starts with initial number of clusters equal to the total number of speaker segments.…”

Section: Speaker Clusteringmentioning

confidence: 99%

“…On the other hand, utterances from same speakers must not be distributed among multiple clusters. Several approaches to speaker clustering task exist, for example cost optimization, sequential and Agglomerative Hierarchical Clustering [12,13,14,15]. Some approaches rely on commonly used statistical speaker modeling like Gaussian Mixture Models (GMMs) while others use features extracted using Deep Neural Networks (DNNs).…”

Section: Introductionmentioning

confidence: 99%

Restricted Boltzmann Machine Vectors for Speaker Clustering

Khan¹,

Safari²,

Hernando³

2018

IberSPEECH 2018

View full text Add to dashboard Cite

Restricted Boltzmann Machines (RBMs) have been used both in the front-end and backend of speaker verification systems. In this work, we apply RBMs as a front-end in the context of speaker clustering. Speakers' utterances are transformed into a vector representation by means of RBMs. These vectors, referred to as RBM vectors, have shown to preserve speakerspecific information and are used for the task of speaker clustering. In this work, we perform the traditional bottom-up Agglomerative Hierarchical Clustering (AHC). Using the RBM vector representation of speakers, the performance of speaker clustering is improved. The evaluation has been performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed system outperforms the baseline i-vectors system in terms of Equal Impurity (EI). Using cosine scoring, a relative improvement of 11% and 12% are achieved for average and single linkage clustering algorithms respectively. Using PLDA scoring, the RBM vectors achieve a relative improvement of 11% compared to i-vectors for the single linkage algorithm.

show abstract

A study of speaker clustering for speaker attribution in large telephone conversation datasets

Cited by 11 publications

References 29 publications

Semi-supervised acoustic model training for speech with code-switching

Semi-supervised acoustic model training for speech with code-switching

Large-Scale Speaker Diarization of Radio Broadcast Archives

Restricted Boltzmann Machine Vectors for Speaker Clustering

Contact Info

Product

Resources

About