Deep Learning Based Dereverberation of Temporal Envelopesfor Robust Speech Recognition

Purushothaman, Anurenjan; Sreeram, Anirudh; Kumar, Rohit; Ganapathy, Sriram

doi:10.48550/arxiv.2008.03339

Cited by 2 publications

(6 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use FDLP features [31] for far-field speech. This paper extends the prior work done in [32] by proposing a joint neural dereverberation which forms an elegant neural learning framework. Further, several ASR experiments with the joint modeling approach are also conducted in this work.…”

Section: Related Prior Workmentioning

confidence: 76%

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

Purushothaman¹,

Sreeram²,

Kumar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The task of speech recognition in far-field environments is adversely affected by the reverberant artifacts that elicit as the temporal smearing of the sub-band envelopes. In this paper, we develop a neural model for speech dereverberation using the long-term sub-band envelopes of speech. The sub-band envelopes are derived using frequency domain linear prediction (FDLP) which performs an autoregressive estimation of the Hilbert envelopes. The neural dereverberation model estimates the envelope gain which when applied to reverberant signals suppresses the late reflection components in the far-field signal. The dereverberated envelopes are used for feature extraction in speech recognition. Further, the sequence of steps involved in envelope dereverberation, feature extraction and acoustic modeling for ASR can be implemented as a single neural processing pipeline which allows the joint learning of the dereverberation network and the acoustic model. Several experiments are performed on the REVERB challenge dataset, CHiME-3 dataset and VOiCES dataset. In these experiments, the joint learning of envelope dereverberation and acoustic model yields significant performance improvements over the baseline ASR system based on log-mel spectrogram as well as other past approaches for dereverberation (average relative improvements of 10-24% over the baseline system). A detailed analysis on the choice of hyper-parameters and the cost function involved in envelope dereverberation is also provided.

show abstract

Section: Related Prior Workmentioning

confidence: 76%

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

Purushothaman¹,

Sreeram²,

Kumar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…al [27], however the frequency domain loss is used for training. Previously, we had proposed a convolutional neural network model to perform dereverberation of speech [17,18]. This approach used a neural network applied on autoregressive models of sub-band envelopes to perform dereverberation.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Let x q (n), h q (n) and r q (n) denote the sub-band clean speech, room-response and the reverberant speech respectively for the qth sub-band. The sub-band envelopes of farfield speech m rq (n), extracted using frequency domain linear prediction (FDLP) [28,3], can be approximated as, [17]…”

Section: Signal Modelmentioning

confidence: 99%

“…In this section, we describe the proposed approach to envelope dereverberation. The proposed framework (Figure 1), consists of three modules, (i) Envelope Dereverberation [17], (ii) E2E ASR framework and (iii) Joint Learning.…”

Section: Envelope Dereverberation and E2e Asrmentioning

confidence: 99%

“…Our previous work [17,18] explored the use of dereverberation of sub-band envelopes for hybrid speech recognition systems. The sub-band envelopes are extracted using the autoregressive modeling framework of frequency domain linear prediction [19,20].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

End-to-End Speech Recognition With Joint Dereverberation Of Sub-Band Autoregressive Envelopes

Kumar¹,

Purushothaman²,

Sreeram³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The end-to-end (E2E) automatic speech recognition (ASR) offers several advantages over previous efforts for recognizing speech. However, in reverberant conditions, E2E ASR is a challenging task as the long-term sub-band envelopes of the reverberant speech are temporally smeared. In this paper, we develop a feature enhancement approach using a neural model operating on sub-band temporal envelopes. The temporal envelopes are modeled using the framework of frequency domain linear prediction (FDLP). The neural enhancement model proposed in this paper performs an envelope gain based enhancement of temporal envelopes. The model architecture consists of a combination of convolutional and long short term memory (LSTM) neural network layers. Further, the envelope dereverberation, feature extraction and acoustic modeling using transformer based E2E ASR can all be jointly optimized for the speech recognition task. The joint optimization ensures that the dereverberation model targets the ASR cost function. We perform E2E speech recognition experiments on the REVERB challenge dataset as well as on the VOiCES dataset. In these experiments, the proposed joint modeling approach yields significant improvements compared to baseline E2E ASR system (average relative improvements of 21% on the REVERB challenge dataset and about 10% on the VOiCES dataset).

show abstract

Deep Learning Based Dereverberation of Temporal Envelopesfor Robust Speech Recognition

Cited by 2 publications

References 27 publications

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

End-to-End Speech Recognition With Joint Dereverberation Of Sub-Band Autoregressive Envelopes

Contact Info

Product

Resources

About