Convolutional maxout neural networks for speech separation

A comprehensive assessment of ecosystem dynamics requires the monitoring of biological, physical and social changes. Changes that cannot be observed visually may be trackable acoustically through soundscape analysis. Soundscapes vary greatly depending on geophysical events, biodiversity and human activities. However, retrieving source-specific information from geophony, biophony and anthropophony remains a challenging task, due to interference by simultaneous sound sources. Audio source separation is a technique that aims to recover individual sound sources when only mixtures are accessible. Here, we review techniques of monoaural audio source separation with the fundamental theories and assumptions behind them. Depending on the availability of prior information about the source signals, the task can be approached as a blind source separation or a model-based source separation. Most blind source separation techniques depend on assumptions about the behaviour of the source signals, and their performance may deteriorate when the assumptions fail. Model-based techniques generally do not require specific assumptions, and the models are directly learned from labelled data. With the recent advances of deep learning, the model-based techniques can yield state-of-the-art separation performance, accordingly facilitate content-based audio information retrieval. Source separation techniques have been adopted in several ecoacoustic applications to evaluate the contributions from biodiversity and anthropogenic disturbance to soundscape dynamics. They can also be employed as nonlinear filters to improve the recognition of bioacoustic signals. To effectively retrieve ecological information from soundscapes, source separation is a crucial tool. We believe that the future integrations of ecological hypotheses and deep learning can realize a high-performance source separation for ecoacoustics, and accordingly improve soundscape-based ecosystem monitoring. Therefore, we outline a roadmap for applying source separation to assist in soundscape information retrieval and hope to promote cross-disciplinary collaboration. (biophony), their habitats (geophony) and human development (anthropophony) (Dumyahn and Pijanowski 2011; Pijanowski et al. 2011; Sueur and Farina 2015; Krause and Farina 2016). By listening to biotic sounds, the composition of soniferous animals can be characterized and applied to assess wildlife biodiversity (Andr e et al. 2011; Saito et al. 2015; Lin et al. 2017a; Ross et al. 2018). On the other hand, abiotic sounds can be used to evaluate the ecosystem dynamics associated with

show abstract

“…), and CNN (Hui et al. ; Nugraha et al. ), are the current state of the art in the separation of speech and music (Stöter et al.…”

Section: Monoaural Audio Source Separationmentioning

confidence: 99%

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

Lin

Tsao

2019

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

show abstract

“…In terms of front-end models, DNNs are the most common approach [6], though RNNs have been used for speech enhancement as well, as in [23]. There have also been a few studies that used CNNs for front-end speech denoising [24,25,26]. In the last of these, the authors used a single "bypass" connection from the encoder to the decoder, but none of the models described here can be said to use residual connections.…”

Section: Prior Workmentioning

confidence: 99%

An Exploration of Mimic Architectures for Residual Network Based Spectral Mapping

Plantinga

Bagchi

Fosler‐Lussier

2018

2018 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Spectral mapping uses a deep neural network (DNN) to map directly from noisy speech to clean speech. Our previous study [1] found that the performance of spectral mapping improves greatly when using helpful cues from an acoustic model trained on clean speech. The mapper network learns to mimic the input favored by the spectral classifier and cleans the features accordingly. In this study, we explore two new innovations: we replace a DNN-based spectral mapper with a residual network that is more attuned to the goal of predicting clean speech. We also examine how integrating long term context in the mimic criterion (via wide-residual biL-STM networks) affects the performance of spectral mapping compared to DNNs. Our goal is to derive a model that can be used as a preprocessor for any recognition system; the features derived from our model are passed through the standard Kaldi ASR pipeline and achieve a WER of 9.3%, which is the lowest recorded word error rate for CHiME-2 dataset using only feature adaptation.

show abstract

“…Instead of using a masking function, the clean speech coefficients have also been used as target of a DNN in many approaches [6], [7], [36], [39]. For this, various architecture of DNNs have been employed, e.g., feed-forward networks [6], [7], recurrent neural networks [24] including long short-term memory cells [37], [40]- [42], generative adversarial networks [43], [44], convolutional neural networks [27], [29], [45] and WaveNet based approaches [28], [46].…”

Section: Introductionmentioning

confidence: 99%

SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement

Rehr,

Gerkmann

2020

Preprint

View full text Add to dashboard Cite

This paper analyzes the generalization of speech enhancement algorithms based on deep neural networks (DNNs) with respect to (1) the chosen features, (2) the size and diversity of the training data, and (3) different network architectures. To address (1), we compare three input features, namely logarithmized noisy periodograms, noise aware training (NAT) and signal-to-noise ratio (SNR) based noise aware training (SNR-NAT). For improving the generalization over noisy periodograms, NAT appends an estimate of the noise power spectral density (PSD) to these features, whereas the proposed SNR-NAT uses the noise PSD estimate for normalization. To address (2), we train networks on the Hu noise corpus (limited size), the CHiME 3 noise corpus (limited diversity) and also propose a large and diverse dataset collected based on freely available sounds. On the one hand, we show that increasing the amount and the diversity of training data helps DNNs to generalize. On the other hand, via experimental results and an analysis using t-distributed stochastic neighbor embedding (t-SNE) we show that SNR-NAT features are robust even if the size and the diversity of the input data are limited. To address (3), we compare a fully-connected feedforward DNN and an long short-term memory (LSTM) and show that the LSTM generalizes better for limited training data and simplistic features.

show abstract

Convolutional maxout neural networks for speech separation

Cited by 28 publications

References 12 publications

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

An Exploration of Mimic Architectures for Residual Network Based Spectral Mapping

SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement

Contact Info

Product

Resources

About