On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks

Abstract

In this paper, we address a sub-topic of the broad domain of audio enhancement, namely musical audio bandwidth extension. We formulate the bandwidth extension problem using deep neural networks, where a band-limited signal is provided as input to the network, with the goal of reconstructing a fullbandwidth output. Our main contribution centers on the impact of the choice of low pass filter when training and subsequently testing the network. For two different state of the art deep architectures, ResNet and U-Net, we demonstrate that when the training and testing filters are matched, improvements in signalto-noise ratio (SNR) of up to 7 dB can be obtained. However, when these filters differ, the improvement falls considerably and under some training conditions results in a lower SNR than the bandlimited input. To circumvent this apparent overfitting to filter shape, we propose a data augmentation strategy which utilizes multiple low pass filters during training and leads to improved generalization to unseen filtering conditions at test time.

TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining

“…The performance of BWE models is highly sensitive to different anti-aliasing filters when downsampling methods in testing differ from training [5,6,10]. Similar to [10], to improve the robustness of our model, we generate the low-rate signals by downsampling the high-rate speech dataset with random anti-aliasing filters. More specifically, we adopt the Chebyshev Type I anti-aliasing filter and randomize its ripple and order parameters.…”

“…In addition, training endto-end BWE models requires high-rate target signals, making valuable low-rate data collected from telephony 8-kHz infrastructure unusable. It has also been observed that BWE models are susceptible to low-pass filtering [6,10], generating severe distortion at the transition band of the anti-aliasing filter. This problem can be mitigated by data augmentation [10].…”

“…It has also been observed that BWE models are susceptible to low-pass filtering [6,10], generating severe distortion at the transition band of the anti-aliasing filter. This problem can be mitigated by data augmentation [10].…”

“…The advantage of this pretraining is that it requires only low-rate data to make full use of telephony databases, allowing the model to learn the underlying statistics of the low-band speech and generalize to downstream tasks [12]. Finally, similar to [10], we make our model robust to downsampling methods by generating training data with different parameter sets of the Chebyshev Type I filter.…”

VoiceFixer: Toward General Speech Restoration with Neural Vocoder

“…For example, when microphones have low responses in high-frequency, or audio recordings are compressed to low sampling rates, the high frequencies information will be lost. We follow the description in to produce low-resolution distortions but add more filter types (Sulun & Davies, 2020). After designing a low pass filter h, we first convolve it with s to avoid the aliasing phenomenon.…”