Cam: Context-Aware Masking for Robust Speaker Verification

Yu, Ya-Qi; Zheng, Siqi; Suo, Hongbin; Lei, Yun; Li, Wu-Jun

doi:10.1109/icassp39728.2021.9414704

Cited by 10 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Masked filtering allows us to extract relatively clean embeddings from overlapping speech. This is well illustrated in our previous work [19]. Second, if there is only one speaker in the entire segment, it is of our best interest to include as much information as possible.…”

Section: "Winner Takes All" Masked Filteringmentioning

confidence: 61%

“…The mask prediction network is trained on the same setup as described in [19]. The only difference is that, instead of taking a rough mean pooling as the target embedding, we conduct a 2-class clustering and only run mean pooling on the dominating class.…”

Section: Methodsmentioning

confidence: 99%

“…Features from the dominating component are used as the target embedding. The masked prediction network is trained in the same manner as described in [19].…”

Section: "Winner Takes All" Masked Filteringmentioning

confidence: 99%

See 2 more Smart Citations

Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

Zheng

Suo

2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Clustering-based speaker diarization has stood firm as one of the major approaches in reality, despite recent development in endto-end diarization. However, clustering methods have not been explored extensively for speaker diarization. Commonly-used methods such as k-means, spectral clustering, and agglomerative hierarchical clustering only take into account properties such as proximity and relative densities. In this paper we propose to view clustering-based diarization as a community detection problem. By doing so the topological structure is considered. This work has four major contributions. First it is shown that Leiden community detection algorithm significantly outperforms the previous methods on the clustering of speaker-segments. Second, we propose to use uniform manifold approximation to reduce dimension while retaining global and local topological structure. Third, a masked filtering approach is introduced to extract "clean" speaker embeddings. Finally, the community structure is applied to an end-to-end post-processing network to obtain diarization results. The final system presents a relative DER reduction of up to 70 percent. The breakdown contribution of each component is analyzed.

show abstract

Section: "Winner Takes All" Masked Filteringmentioning

confidence: 61%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

Zheng

Suo

2022

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…1) External adjustment: In ASV, models with a "BN-ReLU-TDNN" structure are often used to achieve better performance [32], [36]. However, BN and TDNN are separated by a non-linear activation function which makes the two linear operators can not be combined in one sequential layer [32].…”

Section: B Re-parameterization For Tmsmentioning

confidence: 99%

TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding

Zhang¹,

Jiang²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Speaker embedding is an important front-end module to explore discriminative speaker features (e.g., X-vector) for many speech applications where speaker information is needed. Current state-of-the-art backbone networks for speaker embedding are designed to aggregate multi-scale features from an utterance with multi-branch network architectures for speaker representation (e.g., ECAPA-TDNN). However, naively adding many branches of multi-scale features with the simple fully convolutional operation could not efficiently improve the performance due to the rapid increase of model parameters and computational complexity. Therefore, in the most current state-of-the-art network architectures, only a few branches corresponding to a limited number of temporal scales could be designed for speaker embeddings. To address this problem, in this paper, we propose an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker embedding network almost without increasing computational costs. The new model is based on the conventional time-delay neural network (TDNN), where the network architecture is smartly separated into two modeling operators: a channel-modeling operator and a temporal multi-branch modeling operator. Adding temporal multi-scale in the temporal multi-branch operator needs only a little bit increase of the number of parameters, and thus save more computational budget for adding more branches with large temporal scales. Moreover, after the model was trained, in the inference stage, we further developed a systemic reparameterization method to convert the multi-branch network topology into a single-path-based topology in order to increase inference speed. We investigated the performance of the new TMS method for automatic speaker verification (ASV) on indomain (VoxCeleb) and out-of-domain (CNCeleb) conditions. Results show that the model based on the TMS method obtained a significant increase in the performance over the state-of-the-art ASV models, i.e., ECAPA-TDNN, and meanwhile, had a better model generalization. Moreover, the proposed model achieved a 29% -46% speed up in inference compared to the state-of-theart ECAPA-TDNN.

show abstract

“…The task of speaker verification aims to determine whether a speaker and a registered identity are the same person [1]. Currently, the mainstream approach to speaker verification relies on deep learning techniques [2][3][4][5][6]. This method initially employs deep learning to extract speaker embeddings, and then estimates the similarity score between the speaker and the registered identity using cosine similarity, thereby determining whether they are the same speaker [7].…”

Section: Introductionmentioning

confidence: 99%

A new speaker verification model integrated with texture variational features

Zhang,

Sun,

Sun

et al. 2024

Preprint

View full text Add to dashboard Cite

Different speakers exhibit unique acoustic characteristics, presenting distinct texture patterns in spectrograms. Current research primarily focused on speaker clustering and the design of speaker verification network architectures, but the potential effects of employing varying degrees of differentiation on speaker characteristics have not been explored in the context of speaker verification tasks. In this study, we drew inspiration from the application of edge-oriented convolution block for image texture extraction in image super-resolution analysis, and proposed a new speaker verification model integrated with texture variational features which designed a feature extraction module called order-FCM. We employed Sobel and Laplacian operators to compute first- and second-order differences in the time-frequency domain, aiming to enhance the network's learning of the richer dynamic properties and higher-order features of speech signals. Additionally, we introduced the global response normalization method from the field of computer vision to normalize global channel features at the end of order-FCM, mitigating potential feature collapse phenomena. We integrated the designed module with existing speaker verification networks, and experimental results demonstrated that the performance of this method significantly surpasses that of baseline models on validation sets provided by Vox1-O, Vox1-E, Vox1-H, and the 2023 International Speaker Recognition Challenge (Voxsrc2023).

show abstract

Cam: Context-Aware Masking for Robust Speaker Verification

Cited by 10 publications

References 26 publications

Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

Reformulating Speaker Diarization As Community Detection With Emphasis On Topological Structure

TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding

A new speaker verification model integrated with texture variational features

Contact Info

Product

Resources

About