Deep Neural Networks and End-to-End Learning for Audio Compression

Abstract

Recent achievements in end-to-end deep learning have encouraged the exploration of tasks dealing with highly structured data with unified deep network models. Having such models for compressing audio signals has been challenging since it requires discrete representations that are not easy to train with end-to-end backpropagation. In this paper, we present an end-to-end deep learning approach that combines recurrent neural networks (RNNs) within the training strategy of variational autoencoders (VAEs) with a binary representation of the latent space. We apply a reparametrization trick for the Bernoulli distribution for the discrete representations, which allows smooth backpropagation. In addition, our approach allows the separation of the encoder and decoder, which is necessary for compression tasks. To our best knowledge, this is the first end-to-end learning for a single audio compression model with RNNs, and our model achieves a Signal to Distortion Ratio (SDR) of 20.54.

End-to-End Training of Both Translation Models in the Back-Translation Framework

“…sg(c) is the operation that detaches its input, c, from the gradient computation in backpropagation. Therefore, the final output, z, is a binary variable, z ∈ {0, 1}, but it can flow the gradient through the first term, p. As a variant of STE, this binary reparameterization trick is easy to implement and work with other network architectures (Rim et al, 2021). We propose a reparameterization trick for the categorical distribution inspired by this binary reparameterization trick.…”