Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Xu, Chenglin; Rao, Wei; Wu, Jibin; Li, Haizhou

doi:10.1109/taslp.2021.3100682

Cited by 20 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The room impulse responses (RIRs) in MC-Libri2Mix is simulated using pyroomacoustics 2 package. For the room configurations, the length and width of each room are randomly drawn in the range [5,10]m, and the height is selected in the range [3,4]m. The reverberation time (RT 60 ) of the reverberant data ranges from 200ms to 600ms.…”

Section: Experiments and Discussionmentioning

confidence: 99%

See 1 more Smart Citation

L-SpEx: Localized Target Speaker Extraction

Meng¹,

Xu²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Speaker extraction aims to extract the target speaker's voice from a multi-talker speech mixture given an auxiliary reference utterance. Recent studies show that speaker extraction benefits from the location or direction of the target speaker. However, these studies assume that the target speaker's location is known in advance or detected by an extra visual cue, e.g., face image or video. In this paper, we propose an end-toend localized target speaker extraction on pure speech cues, that is called L-SpEx. Specifically, we design a speaker localizer driven by the target speaker's embedding to extract the spatial features, including direction-of-arrival (DOA) of the target speaker and beamforming output. Then, the spatial cues and target speaker's embedding are both used to form a top-down auditory attention to the target speaker. Experiments on the multi-channel reverberant dataset called MC-Libri2Mix show that our L-SpEx approach significantly outperforms the baseline system.

show abstract

Section: Experiments and Discussionmentioning

confidence: 99%

“…* Corresponding author. never stop seeking the engineering solution to confer human's selective attention capability on machines, as there is high demand for various real-world applications, such as speech recognition [2,3] and speaker verification [4,5].…”

Section: Introductionmentioning

confidence: 99%

L-SpEx: Localized Target Speaker Extraction

Meng¹,

Xu²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Equal error rates (EER) and detection cost function (DCF) are applied to evaluate the performance of speaker verification systems ( Xu et al, 2021 ). The evaluation metrics EER and DCF refer to two parameters, which are False Acceptation Rate (FAR) and False Rejection Rate (FFR).…”

Section: Methodsmentioning

confidence: 99%

Few-shot short utterance speaker verification using meta-learning

Wang

Zhao

Yang

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

Short utterance speaker verification (SV) in the actual application is the task of accepting or rejecting the identity claim of a speaker based on a few enrollment utterances. Traditional methods have used deep neural networks to extract speaker representations for verification. Recently, several meta-learning approaches have learned a deep distance metric to distinguish speakers within meta-tasks. Among them, a prototypical network learns a metric space that may be used to compute the distance to the prototype center of speakers, in order to classify speaker identity. We use emphasized channel attention, propagation and aggregation in TDNN (ECAPA-TDNN) to implement the necessary function for the prototypical network, which is a nonlinear mapping from the input space to the metric space for either few-shot SV task. In addition, optimizing only for speakers in given meta-tasks cannot be sufficient to learn distinctive speaker features. Thus, we used an episodic training strategy, in which the classes of the support and query sets correspond to the classes of the entire training set, further improving the model performance. The proposed model outperforms comparison models on the VoxCeleb1 dataset and has a wide range of practical applications.

show abstract

“…Besides the works on feature learning (extraction), many efforts were made on the construction of back-end classifiers for SV. The typical classifiers mainly included the Cosine Distance (CD) [1], Probabilistic Linear Discriminant Analysis (PLDA) [36], [37], and deep neural network [38].…”

Section: Related Workmentioning

confidence: 99%

Few-Shot Speaker Identification Using Lightweight Prototypical Network With Feature Grouping and Interaction

Chen

Cao

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Although many efforts have been made on decreasing the model complexity for speaker verification, it is still challenging to deploy speaker verification systems with satisfactory result on low-resource terminals. We design a transformation module that performs feature partition and fusion to implement lightweight speaker verification. The transformation module consists of multiple simple but effective operations, such as convolution, pooling, mean, concatenation, normalization, and element-wise summation. It works in a plug-and-play way, and can be easily implanted into a wide variety of models to reduce the model complexity while maintaining the model error. First, the input feature is split into several low-dimensional feature subsets for decreasing the model complexity. Then, each feature subset is updated by fusing it with the inter-feature-subsets correlational information to enhance its representational capability. Finally, the updated feature subsets are independently fed into the block (one or several layers) of the model for further processing. The features that are output from current block of the model are processed according to the steps above before they are fed into the next block of the model. Experimental data are selected from two public speech corpora (namely VoxCeleb1 and VoxCeleb2). Results show that implanting the transformation module into three models (namely AMCRN, ResNet34, and ECAPA-TDNN) for speaker verification slightly increases the model error and significantly decreases the model complexity. Our proposed method outperforms baseline methods on the whole in memory requirement and computational complexity with lower equal error rate. It also generalizes well across truncated segments with various lengths.

show abstract

Target Speaker Verification With Selective Auditory Attention for Single and Multi-Talker Speech

Cited by 20 publications

References 42 publications

L-SpEx: Localized Target Speaker Extraction

L-SpEx: Localized Target Speaker Extraction

Few-shot short utterance speaker verification using meta-learning

Few-Shot Speaker Identification Using Lightweight Prototypical Network With Feature Grouping and Interaction

Contact Info

Product

Resources

About