Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization

Sang, Mufan; Li, Haoqi; Liu, Fang; Arnold, Andrew O.; Li, Wan

doi:10.48550/arxiv.2112.04459

Cited by 1 publication

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address the classcollision problem in the MoCo system, false negative filtering and infoNCE loss re-weighting were proposed based on the well designed experimental analysis. In combination with the ProtoNCE loss, the formulated C3-MoCo system achieved a 8.6% relative improvement over the best contrastive learning based model [39]. And the proposed C3-DINO (with C3-MoCo assisted) speaker embedding system pushed the benchmark Vox2 test set to 2.5% EER, a new record representing a 48.6% relative improvement against the previous SOTA SSL based SV system.…”

Section: Discussionmentioning

confidence: 92%

“…In [29], [43], the authors extended the self-supervised framework to a semi-supervised paradigm where a small portion of the whole dataset is labeled, which provided a good direction to utilize the real-world data. In [39], a selfsupervised regularization term was proposed in addition to the siamese network, an improved SV performance was obtained in the Voxceleb benchmark. In [44], A DINO based speaker embedding system demonstrates the potential of SSL models for SV, a 4.83% EER on Voxceleb1 test set represents the current SOTA performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

C3-DINO: Joint Contrastive and Non-Contrastive Self-Supervised Learning for Speaker Verification

Zhang

2022

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Self-supervised learning (SSL) has drawn an increased attention in the field of speech processing. Recent studies have demonstrated that contrastive learning is able to learn discriminative speaker embeddings in a self-supervised manner. However, base contrastive self-supervised learning (CSSL) assumes that the pairs generated from a view of anchor instance and any view of other instances are all negative, which introduces many false negative pairs in constructing the loss function. The problem is referred as class-collision, which remains as one major issue that impedes the CSSL based speaker verification (SV) systems from achieving better performances. In the meanwhile, studies reveal that negative sample free SSL frameworks perform well in learning speaker or image representations. In this study, we investigate SSL techniques that lead to an improved SV performance. We first analyse the impact of false negative pairs in the CSSL systems. Then, a multi-stage Class-Collision Correction (C3) method is proposed (e.g., false negative pair filtering, InfoNCE re-weighting and ProtoNCE), which leads to the stateof-the-art (SOTA) CSSL based speaker embedding system. On the basis of the pretrained CSSL model, we further propose to employ a negative sample free SSL objective (i.e., DINO) to fine-tune the speaker embedding network. The resulting speaker embedding system (C3-DINO) achieves 2.5% Equal Error Rate (EER) with a simple Cosine Distance Scoring (CDS) method on Voxceleb1 test set, which outperforms the previous SOTA SSL system (4.86%) by a significant +45% relative improvement. With speaker clustering and pseudo labeling on Voxceleb2 training set, a LDA/CDS back-end applying on the C3-DINO speaker embeddings is able to further push the EER to 2.2%. Comprehensive experimental investigations of the Voxceleb benchmarks and our internal dataset demonstrate the effectiveness of our proposed methods, and the performance gap between the SSL SV and the supervised counterpart narrows further.

show abstract

Section: Discussionmentioning

confidence: 92%