Dereverberation binaural source separation using deep learning

Abstract

This paper reported a deep-learning based binaural separation using gammatone-frequency cepstral coefficient (GFCC) and multi-resolution cochleagram (MRCG) as spectral features and interaural time difference (ITD) and interaural level difference (ILD) as spatial features. A binary mask was estimated by deep neural network (DNN) binary classifier that used the features as a training data and ideal ratio mask (IRM) as a training target. In the experiment, a male speaker as a target speech at azimuth 0o and a female speaker as a masker speech at azimuth 30o, 20o, 10o, and 5o in rooms with 0.32, 0.47, 0.68, and 0.89 s reverberation time (RT60). As a training process, 50 mixtures were used for each condition experiment. The classifier contained two hidden layers, 200 binary neurons and 50 epoch, and Restricted Boltzmann Machine (RBM) was used as pre-training process. The RBM and the classifier learning rate was 1 up to 0.001 from epoch 1 to epoch 50. The sound quality results indicated by 88% intelligibility of STOI method which means that the separated sound is easily understood by the listener. The MOS value is 2.8 which means the sentence is clear but spectral distortion is slightly annoying.

Preserving the beamforming effect for spatial cue-based pseudo-binaural dereverberation of a single source

“…With the development of Neural Networks (NNs), there has been tremendous improvement in a variety of speech recognition and acoustic signal processing tasks [16]. The binaural dereverberation models in [8], [16] and [17] uses artificial neural network (ANN) for binaural dereverberation preprocessing, the model in [18] uses the recurrent neural network (RNN) and interaural cues for speech enhancement in reverberant noisy conditions, while the models in [19] and [20] use the U-Net (a deep convolutional neural network (CNN)) for dereverberation, but these are monaural models.…”

Intelligent Control Method for the Dynamic Range Compressor: A User Study

“…They have been increasingly commonly used for learning a non-linear mappings between low-level features to high-level ones in both the audio and computer vision domains. They demonstrated promising results in a diverse range of audio signal processing tasks such as classification, de-reverberation [8,9], mixing [10] and synthesis [11]. Recently, DNNs have been used to model DRC [12] where an autoencoder is used to map an un-processed audio to the processed audio and is conditioned on the vector of DRC controls.…”

Profiling musical audio processing effects with deep neural networks

“…Deep learning has demonstrated great utility at such diverse audio signal processing tasks as classification, 13,14 onset detection, 15 source separation, 16 event detection, 17 dereverberation, 18,19 denoising, 20 formant estimation, 21 remixing, 22 and synthesis, [23][24][25][26] as well as dynamic range compression to automate the mastering process. 27 In the area of audio component modeling, deep learning has been used to model tube amplifiers 28 and most recently guitar distortion pedals.…”