Differentiable Pooling for Unsupervised Acoustic Model Adaptation

Abstract

We present a deep neural network (DNN) acoustic model that includes parametrised and differentiable pooling operators. Unsupervised acoustic model adaptation is cast as the problem of updating the decision boundaries implemented by each pooling operator. In particular, we experiment with two types of pooling parametrisations: learned $L_p$-norm pooling and weighted Gaussian pooling, in which the weights of both operators are treated as speaker-dependent. We perform investigations using three different large vocabulary speech recognition corpora: AMI meetings, TED talks and Switchboard conversational telephone speech. We demonstrate that differentiable pooling operators provide a robust and relatively low-dimensional way to adapt acoustic models, with relative word error rates reductions ranging from 5--20% with respect to unadapted systems, which themselves are better than the baseline fully-connected DNN-based acoustic models. We also investigate how the proposed techniques work under various adaptation conditions including the quality of adaptation data and complementarity to other feature- and model-space adaptation methods, as well as providing an analysis of the characteristics of each of the proposed approaches.Comment: 11 pages, 7 Tables, 7 Figures in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, num. 11, 201

Half Gaussian-based wavelet transform for pooling layer for convolution neural network

“…In this paper, we have proposed a new pooling methods based on probability function, Figure 1 describes the block diagram of the pooling layer, the basic component of this layer is feature computation, which is extracted depending on algorithm (1) by calculating the basic statistics, which can be used to compute the weights of each element according to (1) and (2),which are represented average and standard deviation respectively [13,20] The second half of Gaussian function represents the statistics between mean and maximum value, which represents the most important characteristics of the signal. So, the Gaussian is reconstructed for upper half of its function as shown in Figure 2, then most significant statistics are calculated.…”

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

“…[117]- [119], [180], [228], [231], [248], [291] Embedding Hybrid [56], [57], [61], [74], [130], [132], [138], [148], [150], [153], [159], [161], [168], [213], [292] E2E [62], [128], [217] Feature Hybrid [56], [74], [75], [135], [138], [168], [230], [285], [289], [290], [293] Data Hybrid [116], [193] embedding classes. The overall RERR is 9.72% 1 .…”

“…Fig. 6 (middle)), spanning results for all adaptation BSV + ivectors [132] DNNEmb + LHUC [74] DNNEmb + fMLLR [74] DiffPooling + fMLLR [289] LHUC + fMLLR [75] f-LHUC + LHUC [153] f-LHUC + ivectors [153] ivectors + LHUC [168] ivectors + fMLLR/VTLN [56] pRELU + fMLLR [290] Relative Error Rate Reduction [%] clusters (cf. Fig.…”

Learning to Pool in Graph Neural Networks for Extrapolation

“…, as substitutes for the max pooling or average pooling used in convolutional neural networks. Similar norm-based pooling functions were used for acoustic modeling (Swietojanski et al, 2016) and text representation (Wu et al, 2020). Compared to GNP, these pooling methods cannot express the sum pooling.…”