Deep Scattering Power Spectrum Features for Robust Speech Recognition

Abstract

Deep scattering spectrum consists of a cascade of wavelet transforms and modulus non-linearity. It generates features of different orders, with the first order coefficients approximately equal to the Mel-frequency cepstrum, and higher order coefficients recovering information lost at lower levels. We investigate the effect of including the information recovered by higher order coefficients on the robustness of speech recognition. To that end, we also propose a modification to the original scattering transform tailored for noisy speech. In particular, instead of the modulus non-linearity we opt to work with power coefficients and, therefore, use the squared modulus non-linearity. We quantify the robustness of scattering features using the word error rates of acoustic models trained on clean speech and evaluated using sets of utterances corrupted with different noise types. Our empirical results show that the second order scattering power spectrum coefficients capture invariants relevant for noise robustness and that this additional information improves generalization to unseen noise conditions (almost 20% relative error reduction on AURORA4). This finding can have important consequences on speech recognition systems that typically discard the second order information and keep only the first order features (known for emulating MFCC and FBANK values) when representing speech.

Multi-Stream Acoustic Modelling Using Raw Real and Imaginary Parts of the Fourier Transform

“…Direct employment of the raw signal representations with minimal or no information loss, along with powerful learning architectures can effectively tackle the aforementioned issue. This has led to successful single-stream acoustic modelling using raw waveforms [4]- [12], raw magnitude [13], deep scattering spectrum [14] and raw phase spectrum [15] with better or comparable performance to the classic features across various tasks, even for databases as small as TIMIT [16]- [18].…”

Wearable Electrocardiogram Quality Assessment Using Wavelet Scattering and LSTM

“…These preprocessing methods provided an (Anden and Mallat, 2014). The wavelet scattering method has been wildly used for acoustic scene classification (Li et al, 2019), speech recognition (Fousek et al, 2015;Joy et al, 2020), and heart sound classification (Mei et al, 2021), which yielded efficient representations for audio processing. However, wavelet scattering currently was seldom used in ECG analysis and application.…”

“…First-order scattering coefficients characterize persistent phenomena such as tendency and envelope, while second-order scattering coefficients characterize transient phenomena such as shock signals and amplitude modulation ( Anden and Mallat, 2014 ). The wavelet scattering method has been wildly used for acoustic scene classification ( Li et al, 2019 ), speech recognition ( Fousek et al, 2015 ; Joy et al, 2020 ), and heart sound classification ( Mei et al, 2021 ), which yielded efficient representations for audio processing. However, wavelet scattering currently was seldom used in ECG analysis and application.…”

Towards Robust Waveform-Based Acoustic Models

“…The latter is challenging to address for waveform-based models due to the fact that the feature extraction process is fully automated and utterance level mean-normalization cannot be performed as in the case of standard non-adaptive filterbank features. Prior work has established that such normalizations can be fundamental in dealing with spurious correlations introduced by different microphones and stationary signal corruptions [2,16].…”