Bayesian Multichannel Speech Enhancement with a Deep Speech Prior

Sekiguchi, Kouhei; Bando, Yoshiaki; Yoshii, Kazuyoshi; Kawahara, Tatsuya

doi:10.23919/apsipa.2018.8659591

Cited by 35 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When noisy environments are covered by training data used for DNN-based mask estimation, MNMF can be initialized by using the results of DNN-based mask estimation [2] to further refine the SCMs of speech and noise. It would be promising to use recently-proposed semi-supervised speech enhancement methods based on NMF or MNMF with a DNN-based prior on speech spectra [51]- [53].…”

Section: Discussionmentioning

confidence: 99%

Unsupervised Speech Enhancement Based on Multichannel NMF-Informed Beamforming for Noise-Robust Automatic Speech Recognition

Shimada

Bando

Mimura

et al. 2019

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

This paper describes multichannel speech enhancement for improving automatic speech recognition (ASR) in noisy environments. Recently, the minimum variance distortionless response (MVDR) beamforming has widely been used because it works well if the steering vector of speech and the spatial covariance matrix (SCM) of noise are given. To estimating such spatial information, conventional studies take a supervised approach that classifies each time-frequency (TF) bin into noise or speech by training a deep neural network (DNN). The performance of ASR, however, is degraded in an unknown noisy environment. To solve this problem, we take an unsupervised approach that decomposes each TF bin into the sum of speech and noise by using multichannel nonnegative matrix factorization (MNMF). This enables us to accurately estimate the SCMs of speech and noise not from observed noisy mixtures but from separated speech and noise components. In this paper we propose online MVDR beamforming by effectively initializing and incrementally updating the parameters of MNMF. Another main contribution is to comprehensively investigate the performances of ASR obtained by various types of spatial filters, i.e., time-invariant and variant versions of MVDR beamformers and those of rank-1 and fullrank multichannel Wiener filters, in combination with MNMF. The experimental results showed that the proposed method outperformed the state-of-the-art DNN-based beamforming method in unknown environments that did not match training data.

show abstract

Section: Discussionmentioning

confidence: 99%

Unsupervised Speech Enhancement Based on Multichannel NMF-Informed Beamforming for Noise-Robust Automatic Speech Recognition

Shimada

Bando

Mimura

et al. 2019

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…[2,3,4,5,6]. Recently, deep generative speech models based on variational autoencoders (VAEs) [7] have been investigated for single-channel [8,9,10,11] and multichannel speech enhancement [12,13,14]. A pre-trained deep generative speech model is combined with a nonnegative matrix factorization (NMF) [15] noise model whose parameters are estimated at test time, from the observation of the noisy mixture signal only.…”

Section: Introductionmentioning

confidence: 99%

A Recurrent Variational Autoencoder for Speech Enhancement

Leglaive

Alameda-Pineda

Girin

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE). The deep generative speech model is trained using clean speech signals only, and it is combined with a nonnegative matrix factorization noise model for speech enhancement. We propose a variational expectationmaximization algorithm where the encoder of the RVAE is finetuned at test time, to approximate the distribution of the latent variables given the noisy speech observations. Compared with previous approaches based on feed-forward fully-connected architectures, the proposed recurrent deep generative speech model induces a posterior temporal dynamic over the latent variables, which is shown to improve the speech enhancement results.Index Terms-Speech enhancement, recurrent variational autoencoders, nonnegative matrix factorization, variational inference.

show abstract

“…both single-channel [8][9][10] and multichannel scenarios [11][12][13][14][15]. The main idea of these studies is to use variational autoencoders (VAEs) [16] to replace the NMF generative model, thus benefiting from DNNs' representational power.…”

Section: Introductionmentioning

confidence: 99%

“…In [8][9][10][11][12], speech enhancement is performed using a pretrained VAE-based generative model of speech spectra combined with an unsupervised NMF model for the noise. Inference of the clean speech involves using the Metropolis-Hastings algorithm [19] to estimate an intractable distribution over the VAE's latent space.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Statistically Principled and Computationally Efficient Approach to Speech Enhancement Using Variational Autoencoders

Pariente¹,

Deleforge²

2019

Interspeech 2019

View full text Add to dashboard Cite

Recent studies have explored the use of deep generative models of speech spectra based of variational autoencoders (VAEs), combined with unsupervised noise models, to perform speech enhancement. These studies developed iterative algorithms involving either Gibbs sampling or gradient descent at each step, making them computationally expensive. This paper proposes a variational inference method to iteratively estimate the power spectrogram of the clean speech. Our main contribution is the analytical derivation of the variational steps in which the encoder of the pre-learned VAE can be used to estimate the variational approximation of the true posterior distribution, using the very same assumption made to train VAEs. Experiments show that the proposed method produces results on par with the aforementioned iterative methods using sampling, while decreasing the computational cost by a factor 36 to reach a given performance.

show abstract

Bayesian Multichannel Speech Enhancement with a Deep Speech Prior

Cited by 35 publications

References 15 publications

Unsupervised Speech Enhancement Based on Multichannel NMF-Informed Beamforming for Noise-Robust Automatic Speech Recognition

Unsupervised Speech Enhancement Based on Multichannel NMF-Informed Beamforming for Noise-Robust Automatic Speech Recognition

A Recurrent Variational Autoencoder for Speech Enhancement

A Statistically Principled and Computationally Efficient Approach to Speech Enhancement Using Variational Autoencoders

Contact Info

Product

Resources

About