Tensor-To-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network

Abstract

We propose a tensor-to-vector regression approach to multi-channel speech enhancement in order to address the issue of input size explosion and hidden-layer size expansion. The key idea is to cast the conventional deep neural network (DNN) based vector-to-vector regression formulation under a tensor-train network (TTN) framework. TTN is a recently emerged solution for compact representation of deep models with fully connected hidden layers. Thus TTN maintains DNN's expressive power yet involves a much smaller amount of trainable parameters. Furthermore, TTN can handle a multi-dimensional tensor input by design, which exactly matches the desired setting in multi-channel speech enhancement. We first provide a theoretical extension from DNN to TTN based regression. Next, we show that TTN can attain speech enhancement quality comparable with that for DNN but with much fewer parameters, e.g., a reduction from 27 million to only 5 million parameters is observed in a single-channel scenario. TTN also improves PESQ over DNN from 2.86 to 2.96 by slightly increasing the number of trainable parameters. Finally, in 8-channel conditions, a PESQ of 3.12 is achieved using 20 million parameters for TTN, whereas a DNN with 68 million parameters can only attain a PESQ of 3.06. Code is available online 1 .

Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement

“…The matrix associated with a DNN hidden layer corresponds to two matrices given the ranks, and the DNN input vector is reshaped into a higher-order input tensor. We have shown that the TT decomposition can keep the representation power of DNN [17]. In [17], we have also demonstrated that for a tensor-to-vector function…”

“…We have shown that the TT decomposition can keep the representation power of DNN [17]. In [17], we have also demonstrated that for a tensor-to-vector function…”

“…Besides, TT-DNN is a compact representation for a fully-connected (FC) layers of DNN into a tensor-train (TT) format [16]. In [17], we were the first to attempt a tensor-train deep neural network (TT-DNN) to tackle the multi-channel speech enhancement task and also demonstrate that the TT representation of a DNN does not cause the quality degradation of the enhanced speech, and it also results in a significant reduction of the model parameters. More importantly, the quality of speech enhancement can be improved over the DNN counterpart by allowing the TT-DNN parameters to grow.…”

Variational Inference-Based Dropout in Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

“…Since it has been shown that RNNs overfit very quickly [11], various regularization methods, such as early stopping or small and under-specified models [12], have to be used during the RNN training stage. Although dropout is normally taken as a simple and effective regularization to overcome the problem of overfitting in deep neural networks [13,14], it has been concluded that the naive dropout regularization to recurrent weights in RNNs cannot reliably solve the RNN overfitting problem because noise added in the recurrent connections leads to model instabilities [15].…”

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

“…Thus, this paper aims at bridging this gap. In particular, we investigate MAE and MSE in terms of performance error bounds and robustness against various noises in the context of the deep neural network (DNN) based vector-to-vector regression, since DNNs offer better representation power and generalization capability in large-scale regression problems, such as those addressed in [18]- [21].…”