WHAMR!: Noisy and Reverberant Single-Channel Speech Separation

Abstract

While significant advances have been made in recent years in the separation of overlapping speech signals, studies have been largely constrained to mixtures of clean, near-field speech, not representative of many real-world scenarios. Although the WHAM! dataset introduced noise to the ubiquitous wsj0-2mix dataset, it did not include the addition of reverberation, generally present in indoor recordings outside of recording studios. The spectral smearing caused by reverberation can result in significant performance degradation for standard deep learning-based speech separation systems, which rely on spectral structure and the sparsity of speech signals to tease apart sources. To address this, we introduce WHAMR!, an augmented version of WHAM! with synthetic reverberated sources, and provide a thorough baseline analysis of current techniques as well as novel cascaded architectures on the newly introduced conditions.

Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism

“…The evaluation is performed on the WHAMR! dataset [21], which consists of simulated noisy and reverberant 2-speaker mixtures. WHAMR!…”

DBnet: Doa-Driven Beamforming Network for end-to-end Reverberant Sound Source Separation

“…pMnet takes the log magnitude spectrum of the beamformer output signals and generates real-valued time-frequency masks corresponding to each source. Motivated by the results in [1,4] where a masking-based network with recurrent layers was used for speech separation, we consider a recurrent pMnet consisting of four BLSTM layers followed by one fully connected layer to estimate the stacked masks corresponding to the sources. As an alternative to the recurrent pMnet, we also consider a convolutional-recurrent pMnet with an encoder-decoder network architecture (see Fig.…”

“…Each sequence was a random 10 s sub-portion from the 30 s signals. To improve the network generalization, we use gradient clipping technique with a maximum L2 norm of 5, similarly as used in [4].…”

Training Noisy Single-Channel Speech Separation with Noisy Oracle Sources: A Large Gap and a Small Step

“…Without access to the clean ground truth s clean k (t), the network is instead trained with the noisy speech signals s noisy k (t) as target. Though arguably more desirable for the separation network to produce s clean k (t) than s noisy k (t), it is not an inherently incorrect separation solution as long as the speech signals themselves have been separated, particularly when taking into account the possibility to then feed the noisy separated output into a de-noising speech enhancement network, something shown to be successful with synthetic noisy mixtures [8]. However, this paradigm nevertheless likely has issues stemming from inseparability of the noise mixtures.…”