End-to-End Multi-Speaker Speech Recognition Using Speaker Embeddings and Transfer Learning

Denisov, Pavel; Vu, Ngoc Thang

doi:10.21437/interspeech.2019-1130

Cited by 24 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TL as an advanced variant of ML has attained great success in various fields, e.g., speech recognition [8,9], text mining [10], computer vision [11,12], and ubiquitous computing [13,14] over the last two decades. The existing TL approaches are categorized into following three main groups: (1) instance-based [15], (2) modelbased [16,17] and (3) feature-based [18,19] approaches.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Domain Adaptation via Bregman divergence minimization

2021

View full text Add to dashboard Cite

In recent years, the Fisher linear discriminant analysis (FLDA) based classification models are among the most successful approaches and have shown effective performance in different classification tasks. However, when the learning data (source domain) have a different distribution compared with the testing data (target domain), the FLDA-based models may not work well, and the performance degrades, dramatically. To face this issue, we offer an optimal domain adaptation via Bregman divergence minimization (DAB) approach, in which the discriminative features of source and target domains are simultaneously learned via domain invariant representation. DAB is designed based on the constraints of FLDA, with the aim of the coupled marginal and conditional distribution shifts adaptation through Bregman divergence minimization. Thus, the resulting representation can show well functionality like FLDA and simultaneously discriminate across various classes, as well. Moreover, our proposed approach can be easily kernelized to deal with nonlinear tasks. Different experiments on various benchmark datasets demonstrate that our DAB can constructively face with the cross domain divergence and outperforms other novel state-of-the-art domain adaptation approaches in crossdistribution domains.

show abstract

Section: Related Workmentioning

confidence: 99%

“…where tr −2 (X) displays the inverse of tr(X) square root. [39], which belongs to two Gaussian kernels, due to the quadratic form of equations ( 7) and (8), where the derivative of D(W, X S , X T ) considering W is:…”

Section: Objective Functionmentioning

confidence: 99%

Domain Adaptation via Bregman divergence minimization

2021

View full text Add to dashboard Cite

show abstract

“…In our experiments, we first explore multi-condition training, which involves pooling training samples from different conditions and training them simultaneously [32]. In multi-condition training, we train one model using pooled CS and monolingual data.…”

Section: Transfer Learningmentioning

confidence: 99%

Investigations on Speech Recognition Systems for Low-Resource Dialectal Arabic-English Code-Switching Speech

Hamed¹,

Denisov²,

Li³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Code-switching (CS), defined as the mixing of languages in conversations, has become a worldwide phenomenon. The prevalence of CS has been recently met with a growing demand and interest to build CS ASR systems. In this paper, we present our work on code-switched Egyptian Arabic-English automatic speech recognition (ASR). We first contribute in filling the huge gap in resources by collecting, analyzing and publishing our spontaneous CS Egyptian Arabic-English speech corpus. We build our ASR systems using DNN-based hybrid and Transformer-based end-to-end models. In this paper, we present a thorough comparison between both approaches under the setting of a low-resource, orthographically unstandardized, and morphologically rich language pair. We show that while both systems give comparable overall recognition results, each system provides complementary sets of strength points. We show that recognition can be improved by combining the outputs of both systems. We propose several effective system combination approaches, where hypotheses of both systems are merged on sentence-and word-levels. Our approaches result in overall WER relative improvement of 4.7%, over a baseline performance of 32.1% WER. In the case of intra-sentential CS sentences, we achieve WER relative improvement of 4.8%. Our best performing system achieves 30.6% WER on ArzEn test set.

show abstract

“…Target speech extraction [10] enables the recognition of the target speaker's voice by extracting only his/her voice from the observed mixture. Various approaches have been proposed for speech separation and extraction so far, and even in single-channel setup, they realized drastic improvement on overlapping speech recognition by dealing with interfering speech [5,6,[11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition

et al. 2021

View full text Add to dashboard Cite

Although recent advances in deep learning technology improved automatic speech recognition (ASR), it remains difficult to recognize speech when it overlaps other people's voices. Speech separation or extraction is often used as a front-end to ASR to handle such overlapping speech. However, deep neural network-based speech enhancement can generate 'processing artifacts' as a side effect of the enhancement, which degrades ASR performance. For example, it is well known that single-channel noise reduction for non-speech noise (nonoverlapping speech) often does not improve ASR. Likewise, the processing artifacts may also be detrimental to ASR in some conditions when processing overlapping speech with a separation/extraction method, although it is usually believed that separation/extraction improves ASR. In order to answer the question 'Do we always have to separate/extract speech from mixtures?', we analyze ASR performance on observed and enhanced speech at various noise and interference conditions, and show that speech enhancement degrades ASR under some conditions even for overlapping speech. Based on these findings, we propose a simple switching algorithm between observed and enhanced speech based on the estimated signal-to-interference ratio and signal-to-noise ratio. We demonstrated experimentally that such a simple switching mechanism can improve recognition performance when processing artifacts are detrimental to ASR.

show abstract

End-to-End Multi-Speaker Speech Recognition Using Speaker Embeddings and Transfer Learning

Cited by 24 publications

References 35 publications

Domain Adaptation via Bregman divergence minimization

Domain Adaptation via Bregman divergence minimization

Investigations on Speech Recognition Systems for Low-Resource Dialectal Arabic-English Code-Switching Speech

Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition

Contact Info

Product

Resources

About