3-D Acoustic Modeling for Far-Field Multi-Channel Speech Recognition

Purushothaman, Anurenjan; Sreeram, Anirudh; Ganapathy, Sriram

doi:10.1109/icassp40776.2020.9054481

“…This is the same frequency decomposition used in the FDLP and FDLP-dereverberation experiments. The acoustic model is the 2-D CLSTM network described in Purushothaman et al (2020).…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…We use FDLP features (Purushothaman et al, 2020) for far-field speech. This paper extends the prior work done in (Purushothaman et al, 2020b) by proposing a joint neural dereverberation which forms an elegant neural learning framework.…”

Section: Related Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

Dereverberation of autoregressive envelopes for far-field speech recognition

Purushothaman

¹

,

Sreeram

²

,

Kumar

³

et al. 2022

Computer Speech & Language

Self Cite

View full text Add to dashboard Cite

The task of speech recognition in far-field environments is adversely affected by the reverberant artifacts that elicit as the temporal smearing of the sub-band envelopes. In this paper, we develop a neural model for speech dereverberation using the long-term sub-band envelopes of speech. The sub-band envelopes are derived using frequency domain linear prediction (FDLP) which performs an autoregressive estimation of the Hilbert envelopes. The neural dereverberation model estimates the envelope gain which when applied to reverberant signals suppresses the late reflection components in the far-field signal. The dereverberated envelopes are used for feature extraction in speech recognition. Further, the sequence of steps involved in envelope dereverberation, feature extraction and acoustic modeling for ASR can be implemented as a single neural processing pipeline which allows the joint learning of the dereverberation network and the acoustic model. Several experiments are performed on the REVERB challenge dataset, CHiME-3 dataset and VOiCES dataset. In these experiments, the joint learning of envelope dereverberation and acoustic model yields significant performance improvements over the baseline ASR system based on log-mel spectrogram as well as other past approaches for dereverberation (average relative improvements of 10-24% over the baseline system). A detailed analysis on the choice of hyper-parameters and the cost function involved in envelope dereverberation is also provided.

show abstract

“…We use FDLP features [31] for far-field speech. This paper extends the prior work done in [32] by proposing a joint neural dereverberation which forms an elegant neural learning framework.…”

Section: Related Prior Workmentioning

confidence: 99%

“…The architecture of the acoustic model is based on convolutional long short term memory (CLSTM) networks (Figure 1). The acoustic model corresponds to 2-D CLSTM network described in [31], consisting of 4 layers of CNN, a layer of LSTM with 1024 units performing recurrence over frequency and 3 fully connected layers with batch normalization.…”

Section: Acoustic Modelmentioning

confidence: 99%

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

Purushothaman¹,

Sreeram²,

Kumar³

et al. 2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

The task of speech recognition in far-field environments is adversely affected by the reverberant artifacts that elicit as the temporal smearing of the sub-band envelopes. In this paper, we develop a neural model for speech dereverberation using the long-term sub-band envelopes of speech. The sub-band envelopes are derived using frequency domain linear prediction (FDLP) which performs an autoregressive estimation of the Hilbert envelopes. The neural dereverberation model estimates the envelope gain which when applied to reverberant signals suppresses the late reflection components in the far-field signal. The dereverberated envelopes are used for feature extraction in speech recognition. Further, the sequence of steps involved in envelope dereverberation, feature extraction and acoustic modeling for ASR can be implemented as a single neural processing pipeline which allows the joint learning of the dereverberation network and the acoustic model. Several experiments are performed on the REVERB challenge dataset, CHiME-3 dataset and VOiCES dataset. In these experiments, the joint learning of envelope dereverberation and acoustic model yields significant performance improvements over the baseline ASR system based on log-mel spectrogram as well as other past approaches for dereverberation (average relative improvements of 10-24% over the baseline system). A detailed analysis on the choice of hyper-parameters and the cost function involved in envelope dereverberation is also provided.

show abstract

Deep Learning Based Dereverberation of Temporal Envelopesfor Robust Speech Recognition

Purushothaman,

Sreeram,

Kumar

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Automatic speech recognition in reverberant conditions is a challenging task as the long-term envelopes of the reverberant speech are temporally smeared. In this paper, we propose a neural model for enhancement of sub-band temporal envelopes for dereverberation of speech. The temporal envelopes are derived using the autoregressive modeling framework of frequency domain linear prediction (FDLP). The neural enhancement model proposed in this paper performs an envelop gain based enhancement of temporal envelopes and it consists of a series of convolutional and recurrent neural network layers. The enhanced sub-band envelopes are used to generate features for automatic speech recognition (ASR). The ASR experiments are performed on the REVERB challenge dataset as well as the CHiME-3 dataset. In these experiments, the proposed neural enhancement approach provides significant improvements over a baseline ASR system with beamformed audio (average relative improvements of 21% on the development set and about 11% on the evaluation set in word error rates for REVERB challenge dataset).

show abstract

3-D Acoustic Modeling for Far-Field Multi-Channel Speech Recognition

Cited by 3 publications

References 36 publications

Dereverberation of autoregressive envelopes for far-field speech recognition

Dereverberation of autoregressive envelopes for far-field speech recognition

Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition

Deep Learning Based Dereverberation of Temporal Envelopesfor Robust Speech Recognition

Contact Info

Product

Resources

About