Phasebook and Friends: Leveraging Discrete Representations for Source Separation

Abstract

Deep learning based speech enhancement and source separation systems have recently reached unprecedented levels of quality, to the point that performance is reaching a new ceiling. Most systems rely on estimating the magnitude of a target source by estimating a real-valued mask to be applied to a time-frequency representation of the mixture signal. A limiting factor in such approaches is a lack of phase estimation: the phase of the mixture is most often used when reconstructing the estimated time-domain signal. Here, we propose "magbook", "phasebook", and "combook", three new types of layers based on discrete representations that can be used to estimate complex time-frequency masks. Magbook layers extend classical sigmoidal units and a recently introduced convex softmax activation for mask-based magnitude estimation. Phasebook layers use a similar structure to give an estimate of the phase mask without suffering from phase wrapping issues. Combook layers are an alternative to the magbook-phasebook combination that directly estimate complex masks. We present various training and inference schemes involving these representations, and explain in particular how to include them in an end-to-end learning framework. We also present an oracle study to assess upper bounds on performance for various types of masks using discrete phase representations. We evaluate the proposed methods on the wsj0-2mix dataset, a well-studied corpus for single-channel speaker-independent speaker separation, matching the performance of state-of-theart mask-based approaches without requiring additional phase reconstruction steps.

Phase Reconstruction Based On Recurrent Phase Unwrapping With Deep Neural Networks

“…Motivated by the recent advance in deep learning, several DNNbased phase reconstruction methods have been presented [18][19][20][21][22][23]. However, phase reconstruction from a given amplitude spectrogram is not an easy task for DNNs due to the following two problems: the wrapping effect and sensitivity to a shift of a waveform.…”

“…To take advantage of more knowledge about the target signal, deep neural network (DNN)-based phase reconstruction [18][19][20][21][22][23] has gained increasing attention. Although DNNs have strong modeling capability and learn rich knowledge from training data, DNN-based phase reconstruction has the following two problems: the wrapping effect and sensitivity to a waveform shift.…”

CLCNET: Deep Learning-Based Noise Reduction for Hearing aids using Complex Linear Coding

“…One approach is to estimate magnitude and phase or a phase representation either directly [5] or use the estimated magnitude and noisy phase to predict the clean phase [8]. Estimating the clean phase directly, however, is quite hard, because of its spontaneous, random-like nature.…”

“…Due to a superposition of multiple, quasi static signals within one frequency bin, not only the phase changes over time but also the magnitude as a result of cancellation. Thus, low-res spectrograms limit the effectiveness of standard complex valued processing methods that mainly bring phase improvement [5,8].…”

Real-Time Speech Enhancement Using Equilibriated RNN

“…Recently, speech enhancement is advanced by the use of a deep neural network (DNN) to estimate a T-F mask. For effectively modelling a speech signal which is timesequential data, a recurrent neural network (RNN) is used in various speech signal processing applications [1][2][3][4][5][6][7][8][9][10][11][12][13][14].…”