A Statistically Principled and Computationally Efficient Approach to Speech Enhancement Using Variational Autoencoders

Abstract

Recent studies have explored the use of deep generative models of speech spectra based of variational autoencoders (VAEs), combined with unsupervised noise models, to perform speech enhancement. These studies developed iterative algorithms involving either Gibbs sampling or gradient descent at each step, making them computationally expensive. This paper proposes a variational inference method to iteratively estimate the power spectrogram of the clean speech. Our main contribution is the analytical derivation of the variational steps in which the encoder of the pre-learned VAE can be used to estimate the variational approximation of the true posterior distribution, using the very same assumption made to train VAEs. Experiments show that the proposed method produces results on par with the aforementioned iterative methods using sampling, while decreasing the computational cost by a factor 36 to reach a given performance.

FastMVAE: A Fast Optimization Algorithm for the Multichannel Variational Autoencoder Method

“…To reduce the computational cost, we previously proposed to exploit the pretrained encoder of a CVAE as an approximate posterior estimator to infer the latent space variable z in [1]. With the same motivation, a fast algorithm for estimating the parameters of the VAE-NMF model was later derived based on the Bayesian inference in [38] for single-channel speech enhancement.…”

A Recurrent Variational Autoencoder for Speech Enhancement

“…where ϕ FFNN enc (⋅ ; θenc) ∶ C F ↦ R L × R L + denotes the output of an FFNN. Such an architecture was used in [8,9,10,11,12,13,14]. This is the only case where, from the approximate posterior, we can sample all latent vectors in parallel for all time frames, without further approximation.…”

Robust Unsupervised Audio-Visual Speech Enhancement Using a Mixture of Variational Autoencoders

“…for all TF bins (f, n). Similarly as done in the previous works [5][6][7][8][9][10], we use an unsupervised NMF-based Gaussian noise model that assumes independence across TF bins:…”