Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation

Tak, Hemlata; Todisco, Massimiliano; Wang, Xin; Jung, Jee-weon; Yamagishi, Junichi; Evans, Nicholas

doi:10.48550/arxiv.2202.12233

Cited by 9 publications

(10 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current solutions leverage end-toend deep neural networks (DNNs) [12,13], trying to distinguish artifacts and unnatural cues of spoofing speech from bona fide speech. And thanks to a series of challenges and datasets [1][2][3][4], many novel techniques were introduced to achieve promising CM performances [12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28].…”

Section: Introductionmentioning

confidence: 99%

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Wu¹,

Meng²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recently, many novel techniques have been introduced to deal with spoofing attacks, and achieve promising countermeasure (CM) performances. However, these works only take the standalone CM models into account. Nowadays, a spoofing aware speaker verification (SASV) challenge which aims to facilitate the research of integrated CM and ASV models, arguing that jointly optimizing CM and ASV models will lead to better performance, is taking place. In this paper, we propose a novel multi-model and multi-level fusion strategy to tackle the SASV task. Compared with purely scoring fusion and embedding fusion methods, this framework first utilizes embeddings from CM models, propagating CM embeddings into a CM block to obtain a CM score. In the second-level fusion, the CM score and ASV scores directly from ASV systems will be concatenated into a prediction block for the final decision. As a result, the best single fusion system has achieved the SASV-EER of 0.97% on the evaluation set. Then by ensembling the top-5 fusion systems, the final SASV-EER reached 0.89%.

show abstract

Section: Introductionmentioning

confidence: 99%

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Wu¹,

Meng²,

Kang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…One good example is data augmentation, a method that augments the training set using waveforms processed with codec or other signal processing operators [16,17,18]. Another approach is to replace conventional feature extractors with a pre-trained self-supervised-learning (SSL) DNN [15,19]. This DNN is pre-trained on a huge amount of diverse bona fide speech data through SSL [20], and it is able to extract features robust to various channel conditions, languages, and speakers.…”

Section: Introductionmentioning

confidence: 99%

Investigating Active-learning-based Training Data Selection for Speech Spoofing Countermeasure

Wang¹,

Yamagishi²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Training a spoofing countermeasure (CM) that generalizes to various unseen data is desired but challenging. While methods such as data augmentation and self-supervised learning are applicable, the imperfect CM performance on diverse test sets still calls for additional strategies. This study took the initiative and investigated CM training using active learning (AL), a framework that iteratively selects useful data from a large pool set and fine-tunes the CM. This study compared a few methods to measure the data usefulness and the impact of using different pool sets collected from various sources. The results showed that the AL-based CMs achieved better generalization than our strong baseline on multiple test tests. Furthermore, compared with a top-line CM that simply used the whole data pool set for training, the AL-based CMs achieved similar performance using less training data. Although no single best configuration was found for AL, the rule of thumb is to include diverse spoof and bona fide data in the pool set and to avoid any AL data selection method that selects the data that the CM feels confident in.

show abstract

“…More recent studies investigated the effect of selfsupervised front-ends as a speech spoofing countermeasures [22,23]. Self-supervised learning has established itself as a powerful framework to learn general data representations from unlabeled data [24,25,26].…”

Section: Introductionmentioning

confidence: 99%

“…Self-supervised learning has established itself as a powerful framework to learn general data representations from unlabeled data [24,25,26]. Tak et al [22] used wav2vec 2.0 model as front-end of existing AASIST [17] countermeasure network. They used output features of XLS-R [27] to improve the performance of AASIST.…”

Section: Introductionmentioning

confidence: 99%

Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification

Lee,

Kim,

Koo

et al. 2022

Preprint

View full text Add to dashboard Cite

Text-to-speech and voice conversion studies are constantly improving to the extent where they can produce synthetic speech almost indistinguishable from bona fide human speech. In this regrad, the importance of countermeasures (CM) against synthetic voice attacks of the automatic speaker verification (ASV) systems emerges. Nonetheless, most end-to-end spoofing detection networks are black box systems, and the answer to what is an effective representation for finding artifacts still remains veiled. In this paper, we examine which feature space can effectively represent synthetic artifacts using wav2vec 2.0, and study which architecture can effectively utilize the space. Our study allows us to analyze which attribute of speech signals is advantageous for the CM systems. The proposed CM system achieved 0.31% equal error rate (EER) on ASVspoof 2019 LA evaluation set for the spoof detection task. We further propose a simple yet effective spoofing aware speaker verification (SASV) methodology, which takes advantage of the disentangled representations from our countermeasure system. Evaluation performed with the SASV Challenge 2022 database show 1.08% of SASV EER. Quantitative analysis shows that using the explored feature space of wav2vec 2.0 advantages both spoofing CM and SASV.

show abstract

Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation

Cited by 9 publications

References 54 publications

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Spoofing-Aware Speaker Verification by Multi-Level Fusion

Investigating Active-learning-based Training Data Selection for Speech Spoofing Countermeasure

Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification

Contact Info

Product

Resources

About