Bootstrap Equilibrium and Probabilistic Speaker Representation Learning for Self-Supervised Speaker Verification

Abstract

In this paper, we propose self-supervised speaker representation learning strategies, which comprise of a bootstrap equilibrium speaker representation learning in the front-end and an uncertaintyaware probabilistic speaker embedding training in the back-end. In the front-end stage, we learn the speaker representations via the bootstrap training scheme with the uniformity regularization term. In the backend stage, the probabilistic speaker embeddings are estimated by maximizing the mutual likelihood score between the speech samples belonging to the same speaker, which provide not only speaker representations but also data uncertainty. Experimental results show that the proposed bootstrap equilibrium training strategy can effectively help learn the speaker representations and outperforms the conventional methods based on contrastive learning. Also, we demonstrate that the integrated two-stage framework further improves the speaker verification performance on the VoxCeleb1 test set in terms of EER and MinDCF.INDEX TERMS Speaker verification, self-supervised learning, bootstrap representation learning, probabilistic speaker embedding.

Towards Global Video Scene Segmentation with Context-Aware Transformer

“…Considering that state-of-the-art methods (Rao et al 2020;Chen et al 2021;Mun et al 2022) , where y t = 1 represents the boundary, otherwise y t = 0. To this end, we first introduce a context-aware Transformer encoder to model the contextual information, and then propose a self-supervised learning scheme with shot-to-scene pretext tasks to learn discriminative shot representations for segmentation.…”

“…For example, (Chen et al 2021) presented a self-supervised shot embedding approach to learn a shot representation that maximizes the similarity between nearby shots compared to randomly selected shots. (Mun et al 2022) pre-trained a transformer encoder with pseudo-boundaries, and then fine-tuned the encoder with labeled data. Nevertheless, these methods always adopted sophisticated model architectures, without carefully considering the contextual information of the longterm video.…”

“…Shot Encoder. Following (Rao et al 2020;Chen et al 2021;Wu et al 2022b;Mun et al 2022), we employ a shot encoder f e to encode a shot by capturing its spatio-temporal patterns. Given a shot v t , the representations can be formulated as: f e (v t ).…”

“…Therefore, self-supervised approaches (Chen et al 2020;Roh et al 2021) are widely researched by learning effectiveness representation without relying on costly ground-truth annotations. Therefore, the self-supervised learning methods (Chen et al 2021;Mun et al 2022;Wu et al 2022a) have been designed to employ the pre-training protocols for learning spatio-temporal patterns in video scenes. However, in current self-supervised methods, the strategy of pretext task designs and high-level contextual representation modeling are not well addressed.…”

Training Speaker Embedding Extractors Using Multi-Speaker Audio with Unknown Speaker Boundaries

“…The emergence of self-supervision methods in deep learning has also been applied to training speaker embedding extractors [8,9,10,11,12]. Several approaches have been examined, some of which employ an audiovisual setting [13,14].…”