End-to-End Losses Based on Speaker Basis Vectors and All-Speaker Hard Negative Mining for Speaker Verification

Heo, Hee-Soo; Jung, Jee-weon; Yang, Il-Hyung; Yoon, Sung-Hyun; Shim, Hye-jin; Yu, Ha-Jin

doi:10.21437/interspeech.2019-1986

Cited by 24 publications

(22 citation statements)

References 16 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To consider both inter-class and intra-class covariance, we utilize center loss [19] and speaker basis loss [20] in addition to categorical cross-entropy loss for DNN training. We adopt center loss [19] to minimize intra-class covariance while the embedding in the last hidden layer remains discriminative.…”

Section: Additional Objective Functions For Speaker Embedding Enhancementioning

confidence: 99%

“…where xi refers to embedding of the ith utterance, cy i refers to the center of class yi, and N refers to the size of a mini-batch. Speaker basis loss [20], aims to further maximize interclass covariance. This loss function considers a weight vector between the last hidden layer and a node of the softmax output layer as a basis vector for the corresponding speaker and is formulated as: [21] 11.3 x-vector(w augment) [21] 9.9 RawNet 4.8…”

Section: Additional Objective Functions For Speaker Embedding Enhancementioning

confidence: 99%

See 1 more Smart Citation

RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification

et al. 2019

Self Cite

View full text Add to dashboard Cite

Recently, direct modeling of raw waveforms using deep neural networks has been widely studied for a number of tasks in audio domains. In speaker verification, however, utilization of raw waveforms is in its preliminary phase, requiring further investigation. In this study, we explore end-to-end deep neural networks that input raw waveforms to improve various aspects: front-end speaker embedding extraction including model architecture, pre-training scheme, additional objective functions, and back-end classification. Adjustment of model architecture using a pre-training scheme can extract speaker embeddings, giving a significant improvement in performance. Additional objective functions simplify the process of extracting speaker embeddings by merging conventional two-phase processes: extracting utterance-level features such as i-vectors or x-vectors and the feature enhancement phase, e.g., linear discriminant analysis. Effective back-end classification models that suit the proposed speaker embedding are also explored. We propose an end-toend system that comprises two deep neural networks, one frontend for utterance-level speaker embedding extraction and the other for back-end classification. Experiments conducted on the VoxCeleb1 dataset demonstrate that the proposed model achieves state-of-the-art performance among systems without data augmentation. The proposed system is also comparable to the state-of-the-art x-vector system that adopts data augmentation.

show abstract

Section: Additional Objective Functions For Speaker Embedding Enhancementioning

confidence: 99%

Section: Additional Objective Functions For Speaker Embedding Enhancementioning

confidence: 99%

RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this experiment, we compare the embedding vectors obtained from the proposed joint factor embedding scheme and the conventional x-vector framework along with techniques reported in recent studies including VGG-M, ResNet-34 and end-to-end verification systems [39], [40]. The experimented methods are as follows:…”

Section: ) Comparison Between the Joint Factor Embedding Scheme And mentioning

confidence: 99%

“…• VGG [39]: the performance of the embedding extracted from VGG-M, which is a CNN architecture known to perform well on image and speaker classification, reported in [39], • Generalized end-to-end [40]: the performance of the ResNet-34-based end-to-end speaker verification system trained with the generalized end-to-end loss (6) reported in [39], • All-speaker hard negative mining end-to-end [40]:…”

Section: ) Comparison Between the Joint Factor Embedding Scheme And mentioning

confidence: 99%

Disentangled Speaker and Nuisance Attribute Embedding for Robust Speaker Verification

et al. 2020

View full text Add to dashboard Cite

Over the recent years, various deep learning-based embedding methods have been proposed and have shown impressive performance in speaker verification. However, as in most of the classical embedding techniques, the deep learning-based methods are known to suffer from severe performance degradation when dealing with speech samples with different conditions (e.g., recording devices, emotional states). In this paper, we propose a novel fully supervised training method for extracting a speaker embedding vector disentangled from the variability caused by the nuisance attributes. The proposed framework was compared with the conventional deep learning-based embedding methods using the RSR2015 and VoxCeleb1 dataset. Experimental results show that the proposed approach can extract speaker embeddings robust to channel and emotional variability. INDEX TERMS speech embedding, speaker verification, domain disentanglement, deep learning.

show abstract

“…A slightly modified ResNet was used for modeling the spectrograms, accounting for different stride sizes for time and frequency domains due to high-resolution in the frequency domain, and the number of residual blocks was adjusted to fit the provided ASV2019 physical access dataset. The raw waveform CNN-GRU model, proposed in [17], was used with a few modifications: one less residual block, a different specified input utterance length at training phase to fit the dataset, and additional loss functions for training (center loss [25] and speaker basis loss [26]). This model first extracts 128-dimensional frame-level representations using 1-dimensional convolutional layers.…”

Section: Dnn Architecturementioning

confidence: 99%

Replay Attack Detection with Complementary High-Resolution Information Using End-to-End DNN for the ASVspoof 2019 Challenge

Jung¹,

Shim²,

Heo³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

In this study, we concentrate on replacing the process of extracting hand-crafted acoustic feature with end-to-end DNN using complementary high-resolution spectrograms. As a result of advance in audio devices, typical characteristics of a replayed speech based on conventional knowledge alter or diminish in unknown replay configurations. Thus, it has become increasingly difficult to detect spoofed speech with a conventional knowledge-based approach. To detect unrevealed characteristics that reside in a replayed speech, we directly input spectrograms into an end-to-end DNN without knowledge-based intervention. Explorations dealt in this study that differentiates from existing spectrogram-based systems are twofold: complementary information and high-resolution. Spectrograms with different information are explored, and it is shown that additional information such as the phase information can be complementary. High-resolution spectrograms are employed with the assumption that the difference between a bona-fide and a replayed speech exists in the details. Additionally, to verify whether other features are complementary to spectrograms, we also examine raw waveform and an i-vector based system. Experiments conducted on the ASVspoof 2019 physical access challenge show promising results, where t-DCF and equal error rates are 0.0570 and 2.45 % for the evaluation set, respectively.

show abstract

End-to-End Losses Based on Speaker Basis Vectors and All-Speaker Hard Negative Mining for Speaker Verification

Cited by 24 publications

References 16 publications

RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification

RawNet: Advanced End-to-End Deep Neural Network Using Raw Waveforms for Text-Independent Speaker Verification

Disentangled Speaker and Nuisance Attribute Embedding for Robust Speaker Verification

Replay Attack Detection with Complementary High-Resolution Information Using End-to-End DNN for the ASVspoof 2019 Challenge

Contact Info

Product

Resources

About