Target Speaker Extraction for Multi-Talker Speaker Verification

Rao, Wei; Xu, Chenglin; Chng, Eng Siong; Li, Haizhou

doi:10.21437/interspeech.2019-1410

Cited by 27 publications

(28 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also report three competitive baselines (System 2, 11, and 18) that follow the target speaker extraction-verification (TSE-SV) pipeline [31], where speaker extraction and speaker verification modules are trained separately. Between the zeroeffort baselines and competitive baselines, in particular, between System 17 and 18, we observe the followings: (1) The target speaker extraction front-end greatly improves the SV performance under multi-talker test condition; (2) Among System 2, 11 and 18, the frequency-domain SV systems (SV-F and SV-FA) appear to be more robust than the time-domain counterpart (SV-T).…”

Section: Evaluating Target Speaker Embeddings On Wsj0-2talker Datasetmentioning

confidence: 99%

“…The idea of speaker extraction (SE) followed by speaker verification, i.e., SE-SV [31] pipeline, was previously studied to address speaker verification for multi-talker speech. The SE-SV system extracts the speech of the target speaker in the first stage, and subsequently processes the extracted speech with a standard speaker verification module, such as i-vector PLDA [1]- [3].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Rao

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Speaker verification has been studied mostly under the single-talker condition. It is adversely affected in the presence of interference speakers. Inspired by the study on target speaker extraction, e.g., SpEx, we propose a unified speaker verification framework for both single-and multi-talker speech, that is able to pay selective auditory attention to the target speaker. This target speaker verification (tSV) framework jointly optimizes a speaker attention module and a speaker representation module via multitask learning. We study four different target speaker embedding schemes under the tSV framework. The experimental results show that all four target speaker embedding schemes significantly outperform other competitive solutions for multi-talker speech. Notably, the best tSV speaker embedding scheme achieves 76.0% and 55.3% relative improvements over the baseline system on the WSJ0-2mix-extr and Libri2Mix corpora in terms of equalerror-rate for 2-talker speech, while the performance of tSV for single-talker speech is on par with that of traditional speaker verification system, that is trained and evaluated under the same single-talker condition.

show abstract

Section: Evaluating Target Speaker Embeddings On Wsj0-2talker Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Rao

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Integrating target speaker enhancement to robust SV is still at its early stage. The most relevant work to us is [15], where speakerbeam based target speaker enhancement is evaluated [19], and an i-vector model is employed as the SV model. In contrast with the pipeline system [15], we expect an all-neural network solution with joint training leads to improved SV performance.…”

Section: Introductionmentioning

confidence: 99%

Towards Robust Speaker Verification with Target Speaker Enhancement

Zhang

Weng

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper proposes the target speaker enhancement based speaker verification network (TASE-SVNet), an all neural model that couples target speaker enhancement and speaker embedding extraction for robust speaker verification (SV). Specifically, an enrollment speaker conditioned speech enhancement module is employed as the front-end for extracting target speaker from its mixture with interfering speakers and environmental noises. Compared with the conventional target speaker enhancement models, nontarget speaker/interference suppression should draw additional attention for SV. Therefore, an effective nontarget speaker sampling strategy is explored. To improve speaker embedding extraction with a light-weighted model, a teacher-student (T/S) training is proposed to distill speaker discriminative information from large models to small models. Iterative inference is investigated to address the noisy speaker enrollment problem. We evaluate the proposed method on two SV tasks, i.e., one heavily overlapped speech and the other one with comprehensive noise types in vehicle environments. Experiments show significant and consistent improvements in Equal Error Rate (EER) over the state-of-the-art baselines.

show abstract

“…Previous work that consider overlapped speech for the SV task are limited. A single-channel front-end extractor was proposed in [11] to extract the target speaker's speech based on the enrollment speaker information, which greatly improved the performance of the GMM i-vector system in a pipeline mode.…”

Section: Introductionmentioning

confidence: 99%

A Joint Training Framework of Multi-Look Separator and Speaker Embedding Extractor for Overlapped Speech

Zheng

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In multi-talker cases, overlapped speech degrades the speaker verification (SV) performance dramatically. To tackle this challenging problem, speech separation with multi-channel techniques can be adopted to extract each speaker's signals to improve the SV performance. In this paper, a joint training framework of the front-end multi-look speech separator and the back-end speaker embedding extractor is proposed for multi-channel overlapped speech. To better leverage the complementarity between the speech separator and the speaker embedding extractor, several training strategies are proposed to jointly optimize the two modules. Experimental results show that the proposed joint training framework significantly outperforms the individual SV system by around 52% relative EER reduction. Additionally, the robustness of the proposed framework is further evaluated under different conditions.

show abstract

Target Speaker Extraction for Multi-Talker Speaker Verification

Cited by 27 publications

References 22 publications

Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Towards Robust Speaker Verification with Target Speaker Enhancement

A Joint Training Framework of Multi-Look Separator and Speaker Embedding Extractor for Overlapped Speech

Contact Info

Product

Resources

About