Improving Multi-Scale Aggregation Using Feature Pyramid Module for Robust Speaker Verification of Variable-Duration Utterances

Abstract

Currently, the most widely used approach for speaker verification is the deep speaker embedding learning. In this approach, we obtain a speaker embedding vector by pooling single-scale features that are extracted from the last layer of a speaker feature extractor. Multi-scale aggregation (MSA), which utilizes multiscale features from different layers of the feature extractor, has recently been introduced and shows superior performance for variable-duration utterances. To increase the robustness dealing with utterances of arbitrary duration, this paper improves the MSA by using a feature pyramid module. The module enhances speaker-discriminative information of features from multiple layers via a top-down pathway and lateral connections. We extract speaker embeddings using the enhanced features that contain rich speaker information with different time scales. Experiments on the VoxCeleb dataset show that the proposed module improves previous MSA methods with a smaller number of parameters. It also achieves better performance than state-ofthe-art approaches for both short and long utterances.

RSKNet-MTSP: Effective and Portable Deep Architecture for Speaker Verification

“…In Equation 17, C F R is the cost of missed detection; C F A is the cost of a spurious detection; P T arget is a prior probability of the specified target speaker. In this paper, we set C F R = C F A = 1 and P T arget = 0.01, which is commonly used in relevant experiments [27,23].…”

“…strided convolution), some temporal information is lost due to the reduced time dimensions. To address this problem, multi-scale aggregation methods [21,22,23] are gradually introduced into CNN models. However, these methods usually perform global average pooling (GAP) to aggregate features, leading to information loss of speaker characteristics in both time and frequency dimension.…”

“…However, previous pooling methods only focus on two axes (i.e., time axis and channel axis) based frame-level features, which is not fit for the outputs of two-dimensional CNN which have three axes (i.e., time axis, frequency axis and channel axis). On the other hand, prior works presented lots of multi-scale aggregation methods in order to aggregate speaker information from different time scales in deep CNN architectures [21,22,23]. Nonetheless, these methods usually employ global average pooling, which is hard to make full use of speaker information on time and frequency axis.…”

Rep Works in Speaker Verification

“…Several modifications of the backbone architecture are made to improve the performance. These modifications include adding a channel attention [8], transforming the custom convolution into a multi-scale convolution [9,10], and aggregating multi-layer or multi-stage features [4,11]. However, all these methods above only focus on the improvement of singlebranch structures and neglect a multi-branch way of designing neural networks.…”

Graph attentive feature aggregation for text-independent speaker verification

“…In SV, speaker representations are derived by first extracting the frame-level features and then aggregating them. The network extracting the frame-level features is referred to as trunk network (e.g., convolutional neural network (CNN) and x-vector [1][2][3][4][5][6]). After extracting the frame-level features, various techniques including gated recurrent network (GRU), learnable dictionary encoding (LDE) are used on top of the trunk network to aggregate framelevel features into a single utterance-level feature [7][8][9][10][11][12][13][14][15][16][17][18].…”