Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding

Zhu, Hongning; Lee, Kong Aik; Li, Haizhou

doi:10.21437/interspeech.2021-2210

Cited by 14 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A typical speaker verification process involves two stages: First, a few utterances of the speaker are enrolled, then the identity information extracted from the test utterance is compared with that of the enrolled utterances for verification [1]. ASV researchers have been developing speaker embedding extraction methods [2,3,4] to encode speaker identity information for verification. However, it is likely that the test utterance is not human natural speech but spoofing attacks that try to deceive the ASV system.…”

Section: Introductionmentioning

confidence: 99%

A Probabilistic Fusion Framework for Spoofing Aware Speaker Verification

Zhang¹,

Zhu²,

Duan³

2022

Preprint

View full text Add to dashboard Cite

The performance of automatic speaker verification (ASV) systems could be degraded by voice spoofing attacks. Most existing works aimed to develop standalone spoofing countermeasure (CM) systems. Relatively little work targeted at developing an integrated spoofing aware speaker verification (SASV) system. In the recent SASV challenge, the organizers encourage the development of such integration by releasing official protocols and baselines. In this paper, we build a probabilistic framework for fusing the ASV and CM subsystem scores. We further propose fusion strategies for direct inference and fine-tuning to predict the SASV score based on the framework. Surprisingly, these strategies significantly improve the SASV equal error rate (EER) from 19.31% of the baseline to 1.53% on the official evaluation trials of the SASV challenge. We verify the effectiveness of our proposed components through ablation studies and provide insights with score distribution analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

A Probabilistic Fusion Framework for Spoofing Aware Speaker Verification

Zhang¹,

Zhu²,

Duan³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, introducing attention-based modules has become popular in speaker verification recently. Works like attentive statistical pooling [5], cas pooling [4], self-attentive speaker embedding [6] and serialized multi-layer multi-head attention [7] prove that attention mechanism is effective in aggregating frame-level features.…”

Section: Introductionmentioning

confidence: 99%

“…Transformer, first proposed in [8], is also an attentionbased structure and has made a lot of successes in various areas including natural language processing [9] and computer vision [10,11,12], inspired by which we propose a pooling transformer, PoFormer by introducing transformer to the pooling layer of our speaker verification network to strengthen its capability of capturing information across the whole time domain. Different from [7] which re-designs the inner structure of the attention module and implements the multi-head mechanism serially, we strictly follows [8] where different heads are in parallel, providing a simple but effective pooling transformer.…”

Section: Introductionmentioning

confidence: 99%

Poformer: A simple pooling transformer for speaker verification

Ma¹,

Ding²,

Zhao³

et al. 2021

Preprint

View full text Add to dashboard Cite

Most recent speaker verification systems are based on the extraction of speaker embeddings using a deep neural network. The pooling layer in the network aims to aggregate framelevel features extracted by the backbone. In this paper, we propose a new transformer based pooling structure called Po-Former to enhance the ability of the pooling layer to capture information along the whole time axis. Different from previous works that apply attention mechanism in a simple way or implement the multi-head mechanism in serial instead of in parallel, PoFormer follows the initial transformer structure with some minor modifications like a positional encoding generator, drop path and LayerScale to make the training procedure more stable and to prevent overfitting. Evaluated on various datasets, PoFormer outperforms the existing pooling system with at least a 13.00% improvement in EER and a 9.12% improvement in minDCF.

show abstract