EfficientTDNN: Efficient Architecture Search for Speaker Recognition

Abstract

Convolutional neural networks (CNNs), such as the time-delay neural network (TDNN), have shown their remarkable capability in learning speaker embedding. However, they meanwhile bring a huge computational cost in storage size, processing, and memory. Discovering the specialized CNN that meets a specific constraint requires a substantial effort of human experts. Compared with hand-designed approaches, neural architecture search (NAS) appears as a practical technique in automating the manual architecture design process and has attracted increasing interest in spoken language processing tasks such as speaker recognition. In this paper, we propose EfficientTDNN, an efficient architecture search framework consisting of a TDNNbased supernet and a TDNN-NAS algorithm. The proposed supernet introduces temporal convolution of different ranges of the receptive field and feature aggregation of various resolutions from different layers to TDNN. On top of it, the TDNN-NAS algorithm quickly searches for the desired TDNN architecture via weight-sharing subnets, which surprisingly reduces computation while handling the vast number of devices with various resources requirements. Experimental results on the VoxCeleb dataset show the proposed EfficientTDNN enables approximate 10 13 architectures concerning depth, kernel, and width. Considering different computation constraints, it achieves a 2.20% equal error rate (EER) with 204M multiply-accumulate operations (MACs), 1.41% EER with 571M MACs as well as 0.94% EER with 1.45G MACs. Comprehensive investigations suggest that the trained supernet generalizes subnets not sampled during training and obtains a favorable trade-off between accuracy and efficiency.

Few-Shot Speaker Identification Using Lightweight Prototypical Network With Feature Grouping and Interaction

“…The ECAPA-TDNNLite based method [50] is a lightweight version of the ECAPA-TDNN based method, in which a large model, ECAPA-TDNN, is utilized for enrollment and a small model, ECAPA-TDNNLite, is used for verification. The EfficientTDNN based method [51] uses the NAS technique to design an efficient model to implement lightweight SV. The KD-based method [52] needs to train a large teacher model first, and then the KD technique is adopted to obtain a small student model based on the teacher model for realizing lightweight SV.…”

“…In this section, we compare the proposed lightweight method to six state-of-the-art methods for lightweight SV, including the ECAPA-TDNNLite [50], EfficientTDNN [51], KD-based [52], Thin-ResNet34 [64], Fast-ResNet34 [65], and CSTCTS1dConv (Channel Split Time-Channel-Time Separable 1-dimensional Convolution) [66]. The ECAPA-TDNNLite based method [50] is a lightweight version of the ECAPA-TDNN based method, in which a large model, ECAPA-TDNN, is utilized for enrollment and a small model, ECAPA-TDNNLite, is used for verification.…”

“…The generated small model achieved competitive results with 11.6 million floating-point operations per second. Recently, the NAS technique was applied to design an efficient model (termed EfficientTDNN) which obtained satisfactory error rate with low computational complexity [51].…”