Separated Noise Suppression and Speech Restoration: Lstm-Based Speech Enhancement in Two Stages

Strake, Maximilian; Defraene, Bruno; Fluyt, Kristoff; Tirry, Wouter; Fingscheidt, Tim

doi:10.1109/waspaa.2019.8937222

Cited by 27 publications

(29 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, both fine-tuning variants were not further considered for the experimental evaluation. As an additional reference, we include the two-stage method pre-published in [39], which uses a convolutional LSTM layer [57] in between encoder and decoder of the second-stage network. A further difference to our proposed model is the usage of maximum pooling and upsampling layers instead of the computationally more efficient strided and transposed convolutions in the proposed CED-cSA-tr network.…”

Section: Baseline and Proposed Methodsmentioning

confidence: 99%

“…A further difference to our proposed model is the usage of maximum pooling and upsampling layers instead of the computationally more efficient strided and transposed convolutions in the proposed CED-cSA-tr network. We call the method from [39] LSTM-cMSA+CLED-cSAdu and compare it to the proposed methods in terms of performance and computational complexity.…”

Section: Baseline and Proposed Methodsmentioning

confidence: 99%

“…An analysis of the computational complexity of our proposed LSTM-cMSA+CED-cSA setups and the high-complexity reference LSTM-cMSA+CLED-cSA-du [39] in terms of number of parameters, multiplications, and the average time needed to process one frame (L = 256 samples), resulting in a real-time factor, is presented in Table 4. The frame processing time was measured on an Intel Core i5 machine clocked at 3.4 GHz using our Tensorflow implementation without any further optimizations to speed up inference and relying exclusively on CPU processing.…”

Section: Model Complexitymentioning

confidence: 99%

“…6. Comparing the output of the two singlestage methods LSTM-MSA and LSTM-cMSA (third and fourth spectrogram from the Table 4 Comparison of complexity in terms of number of trainable parameters, multiplications and real-time factor (measured on an Intel Core i5 machine clocked at 3.4 GHz) for our proposed methods and the high-complexity reference pre-published in [39]. Comparisons are given for the second stage, for which the methods differ, and for the total two-stage system top, respectively) shows the higher noise suppression that can be obtained with LSTM-cMSA.…”

Section: Analysis Of the Two-stage Approachmentioning

confidence: 99%

“…+ CLED-cSA-du[39] LSTM-cMSA + CED-cSA-du (new) LSTM-cMSA + CED-cSA-tr (new) Spectrograms of reference and enhanced speech for a male speaker in unseen pub noise at 5 dB SNR.From top to bottom: clean speech, noisy speech, LSTM-MSA, LSTM-cMSA, new LSTM-cMSA+DNN-cSA, proposed LSTM-cMSA+CED-cSA-tr, and high-complexity reference LSTM-cMSA+CLED-cSA-du…”

mentioning

confidence: 99%

See 4 more Smart Citations

Speech enhancement by LSTM-based noise suppression followed by CNN-based speech restoration

Strake

Defraene²,

Fluyt³

et al. 2020

EURASIP J. Adv. Signal Process.

Self Cite

View full text Add to dashboard Cite

Single-channel speech enhancement in highly non-stationary noise conditions is a very challenging task, especially when interfering speech is included in the noise. Deep learning-based approaches have notably improved the performance of speech enhancement algorithms under such conditions, but still introduce speech distortions if strong noise suppression shall be achieved. We propose to address this problem by using a two-stage approach, first performing noise suppression and subsequently restoring natural sounding speech, using specifically chosen neural network topologies and loss functions for each task. A mask-based long short-term memory (LSTM) network is employed for noise suppression and speech restoration is performed via spectral mapping with a convolutional encoder-decoder network (CED). The proposed method improves speech quality (PESQ) over state-of-the-art single-stage methods by about 0.1 points for unseen highly non-stationary noise types including interfering speech. Furthermore, it is able to increase intelligibility in low-SNR conditions and consistently outperforms all reference methods.

show abstract

Section: Baseline and Proposed Methodsmentioning

confidence: 99%

Section: Baseline and Proposed Methodsmentioning

confidence: 99%

Section: Model Complexitymentioning

confidence: 99%

Section: Analysis Of the Two-stage Approachmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Speech enhancement by LSTM-based noise suppression followed by CNN-based speech restoration

Strake

Defraene²,

Fluyt³

et al. 2020

EURASIP J. Adv. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

Speech enhancement using deep complex convolutional neural network (DCCNN) model

Iqbal,

Zhang,

Fahad

et al. 2024

SIViP

View full text Add to dashboard Cite

A Hybrid 1D-CNN-LSTM Technique for WKF-Induced Variability of Multi-Channel GAA NS- and NF-FETs

Dash

Sung

2023

IEEE Access

View full text Add to dashboard Cite

Presently deep learning (DL) techniques are massively used in the semiconductor industry. At the same time, applying a deep learning approach for small datasets is also an immense challenge as larger dataset generation needs more computational time-cost factors for technology computer-aided design (TCAD) simulation. In this paper, to overcome the aforesaid issue, a hybrid DL-aided prediction of electrical characteristics of the multichannel devices induced by work function fluctuation (WKF) with a smaller dataset is proposed. For the first time, an amalgamation approach of two deep learning algorithms (i.e.1D-CNN and LSTM) is implemented for all four channels (1 to 4 channels) of gate-all-around (GAA) silicon Nanosheet and Nanofin MOSFETs (NS-FETs and NF-FETs). The proposed joint learning framework combines a one-dimensional convolutional neural network (1D-CNN) with long short-term memory (LSTM) model. In this architecture, CNN can extract the features efficiently from the input WKF, and LSTM identifies the historical sequence of the captured features of the input regression data. To illustrate the excellency of the proposed approach, a comparative study of our hybrid model along with three individual DL models i.e. 1D-CNN and LSTM including a baseline multilevel perceptron (MLP) model are demonstrated for a promising small dataset (i.e.1100 samples). The results indicate a superior prediction of 1D-CNN-LSTM in terms of root mean square error (RMSE) with an average value of 1.7943 X10 −7 and R 2 Score with an average value of 96.18% within the shortest time span in contrast to the other three algorithms. Finally, it can be quantified according to the evaluation and performance that hybrid methodology not only adopts the complexity of both NS-and NF-FETs, but also estimates the characteristics of all four channels of it efficiently with a smaller dataset, lesser time span, and reduced computational cost.

show abstract

Separated Noise Suppression and Speech Restoration: Lstm-Based Speech Enhancement in Two Stages

Cited by 27 publications

References 21 publications

Speech enhancement by LSTM-based noise suppression followed by CNN-based speech restoration

Speech enhancement by LSTM-based noise suppression followed by CNN-based speech restoration

Speech enhancement using deep complex convolutional neural network (DCCNN) model

A Hybrid 1D-CNN-LSTM Technique for WKF-Induced Variability of Multi-Channel GAA NS- and NF-FETs

Contact Info

Product

Resources

About