Enhancing into the Codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders

Abstract

Audio codecs based on discretized neural autoencoders have recently been developed and shown to provide significantly higher compression levels for comparable quality speech output. However, these models are tightly coupled with speech content, and produce unintended outputs in noisy conditions. Based on VQ-VAE autoencoders with WaveRNN decoders, we develop compressor-enhancer encoders and accompanying decoders, and show that they operate well in noisy conditions. We also observe that a compressor-enhancer model performs better on clean speech inputs than a compressor model trained only on clean speech.

Revisiting Speech Content Privacy

“…Another aspect of content privacy is related to speech codecs and compression. A new generative DNN architecture was introduced in [14] and [15], independently, with different speech technologies in mind. The architecture is a dual-encoder vector quantized variational autoencoder (VQ-VAE) that learns to disentangle speech content and speaker identity information in the speech signal while simultaneously creating a discrete and compressed representation of the speech.…”

“…The architecture is a dual-encoder vector quantized variational autoencoder (VQ-VAE) that learns to disentangle speech content and speaker identity information in the speech signal while simultaneously creating a discrete and compressed representation of the speech. In [14] the goal was to use VQ-VAE to compress the speech signal and enhance it by removing unwanted noise. They measured compression rate as bits/sec as well as human judgements of the enhanced speech naturalness.…”

“…One of the main benefits of disentanglement is that it allows content to be modified separately from other informational factors. In addition, some disentanglement techniques such as VQ-VAE also compress the speech signal, which is beneficial for transmission scenarios [14,15]. The VQ-VAE approach is shown in Figure 1.…”