Learning to Denoise Historical Music

Abstract

We propose an audio-to-audio neural network model that learns to denoise old music recordings. Our model internally converts its input into a time-frequency representation by means of a short-time Fourier transform (STFT), and processes the resulting complex spectrogram using a convolutional neural network. The network is trained with both reconstruction and adversarial objectives on a synthetic noisy music dataset, which is created by mixing clean music with real noise samples extracted from quiet segments of old recordings. We evaluate our method quantitatively on held-out test examples of the synthetic dataset, and qualitatively by human rating on samples of actual historical recordings. Our results show that the proposed method is effective in removing noise, while preserving the quality and details of the original music.

A Survey of Audio Enhancement Algorithms for Music, Speech, Bioacoustics, Biomedical, Industrial, and Environmental Sounds by Image U-Net

“…speech [163]. The music inpainting model of [164], uses U-Net as the generator of the Generative Adversarial Network (GAN); another deep neural network, and splits the complex STFT spectrogram in two channels (real and imaginary) to be given at input of the model, which inpaints successfully the musical records, with pauses as long as 100ms. The inpainting model of [165] uses deep complex U-Net and complex-valued STFT spectrograms as input to estimate the complex-valued Ratio Mask (cRM), to restore the gaps due to hiss, clicks, thumps, and other common additive disturbances from old analog gramophone discs.…”

MicAugment: One-shot Microphone Style Transfer

“…We collect a set of 1000 room reverberation IRs by running a room simulator based on the image-source method on rooms of diverse geometry. Finally, we extract 10000 noise segments from the Speech Commands dataset [19], by applying the same extraction method as in [10], i.e., by searching for short low-energy segments of 100 ms length and replicating them (randomizing the phase) with overlap-and-add to obtain samples of 1 second.…”