Two-Stage Enhancement of Noisy and Reverberant Microphone Array Speech for Automatic Speech Recognition Systems Trained with Only Clean Speech

Wang, Quandong; Sicheng, Wang; Ge, Fengpei; Han, Chang Woo; Lee, Jaewon; Guo, Long; Lee, Chin‐Hui

doi:10.1109/iscslp.2018.8706595

Cited by 10 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We avoid giving the proof of our new results here, but the experimental evidence given in the next section on the speech enhancement task supports our theoretical results in Eq. (2).…”

Section: Tensor-to-vector Regressionmentioning

confidence: 99%

See 1 more Smart Citation

Tensor-To-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network

Wang

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We propose a tensor-to-vector regression approach to multi-channel speech enhancement in order to address the issue of input size explosion and hidden-layer size expansion. The key idea is to cast the conventional deep neural network (DNN) based vector-to-vector regression formulation under a tensor-train network (TTN) framework. TTN is a recently emerged solution for compact representation of deep models with fully connected hidden layers. Thus TTN maintains DNN's expressive power yet involves a much smaller amount of trainable parameters. Furthermore, TTN can handle a multi-dimensional tensor input by design, which exactly matches the desired setting in multi-channel speech enhancement. We first provide a theoretical extension from DNN to TTN based regression. Next, we show that TTN can attain speech enhancement quality comparable with that for DNN but with much fewer parameters, e.g., a reduction from 27 million to only 5 million parameters is observed in a single-channel scenario. TTN also improves PESQ over DNN from 2.86 to 2.96 by slightly increasing the number of trainable parameters. Finally, in 8-channel conditions, a PESQ of 3.12 is achieved using 20 million parameters for TTN, whereas a DNN with 68 million parameters can only attain a PESQ of 3.06. Code is available online 1 .

show abstract

“…We avoid giving the proof of our new results here, but the experimental evidence given in the next section on the speech enhancement task supports our theoretical results in Eq. (2).…”

Section: Tensor-to-vector Regressionmentioning

confidence: 99%

“…Deep neural network (DNN) based speech enhancement [1] has demonstrated state-of-the-art performances in a single-channel setting. It has also been extended to multi-channel speech enhancement with similar high-quality enhanced speech [2]. A recent overview can be found in [3].…”

Section: Introductionmentioning

confidence: 99%

Tensor-To-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network

Wang

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For both training and test datasets, the setting of RIRs was fixed to the same conditions, such as the room size, RT60, and all of the distances and directions. Additional detail about the data simulation procedure can be found in [3,17].…”

Section: Data Preparationmentioning

confidence: 99%

“…The state-of-the-art speech enhancement systems are commonly built with deep neural network (DNN) based vector-to-vector regression models, where inputs are context-dependent log power spectrum (LPS) features of noisy speech and outputs correspond to either clean or enhanced LPS features. Although deep neural network (DNN) based speech enhancement [1,2] has demonstrated the state-ofthe-art performance under a single-channel setting, it can also be extended to scenarios of multi-channel speech enhancement with even better-enhanced speech qualities [3]. The process of both single and multi-channel speech enhancement can be taken as a DNN based vector-to-vector regression aiming at bridging a functional relationship f : Y → X such that the input noisy speech y ∈ Y can be mapped to the corresponding clean speech x ∈ X.…”

Section: Introductionmentioning

confidence: 99%

Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement

Wang

et al. 2020

Interspeech 2020

Self Cite

View full text Add to dashboard Cite

This paper investigates different trade-offs between the number of model parameters and enhanced speech qualities by employing several deep tensor-to-vector regression models for speech enhancement. We find that a hybrid architecture, namely CNN-TT, is capable of maintaining a good quality performance with a reduced model parameter size. CNN-TT is composed of several convolutional layers at the bottom for feature extraction to improve speech quality and a tensor-train (TT) output layer on the top to reduce model parameters. We first derive a new upper bound on the generalization power of the convolutional neural network (CNN) based vector-to-vector regression models. Then, we provide experimental evidence on the Edinburgh noisy speech corpus to demonstrate that, in singlechannel speech enhancement, CNN outperforms DNN at the expense of a small increment of model sizes. Besides, CNN-TT slightly outperforms the CNN counterpart by utilizing only 32% of the CNN model parameters. Besides, further performance improvement can be attained if the number of CNN-TT parameters is increased to 44% of the CNN model size. Finally, our experiments of multi-channel speech enhancement on a simulated noisy WSJ0 corpus demonstrate that our proposed hybrid CNN-TT architecture achieves better results than both DNN and CNN models in terms of better-enhanced speech qualities and smaller parameter sizes.

show abstract

“…In [17,18], a unified DNN-based SD speech separation and enhancement system was proposed to jointly handle both background noise and interfering speech, where the speaker-specific data used for DNN training is about 2 hours. In [19], a two-stage approach was proposed for SD enhancement of far-field microphone array speech collected in reverberant conditions corrupted by interfering speakers and noises, where 5 minutes of speakerspecific data is used. [20] adopted more than 5 minutes of speaker-specific data to train a two-stage single-channel SD speech separation system.…”

Section: Introductionmentioning

confidence: 99%

KL-Divergence Regularized Deep Neural Network Adaptation for Low-Resource Speaker-Dependent Speech Enhancement

Chai

Lee

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a Kullback-Leibler divergence (KLD) regularized approach to adapting speaker-independent (SI) speech enhancement model based on regression deep neural networks (DNNs) to another speaker-dependent (SD) model using a tiny amount of speaker-specific adaptation data. This algorithm adapts the DNN model conservatively by forcing the conditional target distribution estimated from the SD model to be close to that from the SI model. The constraint is realized by adding KLD regularization to our previously proposed maximum likelihood objective function. Experimental results demonstrate that, even with only 10 seconds of SD adaptation data, the proposed framework consistently achieves speech intelligibility improvements under all 15 unseen noise types evaluated and at all signal-to-noise ratio levels for all 8 test speakers from the WSJ0 evaluation set.

show abstract

Two-Stage Enhancement of Noisy and Reverberant Microphone Array Speech for Automatic Speech Recognition Systems Trained with Only Clean Speech

Cited by 10 publications

References 37 publications

Tensor-To-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network

Tensor-To-Vector Regression for Multi-Channel Speech Enhancement Based on Tensor-Train Network

Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement

KL-Divergence Regularized Deep Neural Network Adaptation for Low-Resource Speaker-Dependent Speech Enhancement

Contact Info

Product

Resources

About