Phase-Aware Music Super-Resolution Using Generative Adversarial Networks

Hu, Shichao; Zhang, Bin; Liang, Beici; Zhao, Ethan; Lui, Simon

doi:10.21437/interspeech.2020-2605

Cited by 17 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although only a few studies have applied GAN models to bandwidth extension of music signals [10], [39], many recent works have applied them for speech [14], [13], [40]. Eskimez et al [40] proposed one of the earliest works using an adversarial approach for speech super-resolution.…”

Section: B Gans For Audio Bandwidth Extensionmentioning

confidence: 99%

“…However, the Eskimez model had the limitation that it did not predict the phase information but just replicated it [40]. Other phase-aware works made an effort to incorporate the phase information into the training framework [10]. Instead, Kim et al [39], opted for working directly on raw audio, thus avoiding the aforementioned phase issues.…”

Section: B Gans For Audio Bandwidth Extensionmentioning

confidence: 99%

“…During recent years, many works have used modern deep learning technologies for bandwidth extension, but usually with the final goal of increasing the sampling rate of modern digital audio signals. Only a few exceptions are relevant to music signal processing [8], [9], [10], whereas most of these studies focus on processing speech [11], [12], [13], [14], [15].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networks

Moliner¹,

Välimäki²

2022

Preprint

View full text Add to dashboard Cite

Audio bandwidth extension aims to expand the spectrum of narrow-band audio signals. Although this topic has been broadly studied during recent years, the particular problem of extending the bandwidth of historical music recordings remains an open challenge. This paper proposes BEHM-GAN, a model based on generative adversarial networks, as a practical solution to this problem. The proposed method works with the complex spectrogram representation of audio and, thanks to a dedicated regularization strategy, can effectively extend the bandwidth of out-of-distribution real historical recordings. The BEHM-GAN is designed to be applied as a second step after denoising the recording to suppress any additive disturbances, such as clicks and background noise. We train and evaluate the method using solo piano classical music. The proposed method outperforms the compared baselines in both objective and subjective experiments. The results of a formal blind listening test show that BEHM-GAN significantly increases the perceptual sound quality in early-20th-century gramophone recordings. For several items, there is a substantial improvement in the mean opinion score after enhancing historical recordings with the proposed bandwidthextension algorithm. This study represents a relevant step toward data-driven music restoration in real-world scenarios.

show abstract

Section: B Gans For Audio Bandwidth Extensionmentioning

confidence: 99%

Section: B Gans For Audio Bandwidth Extensionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networks

Moliner¹,

Välimäki²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Learning-based methods perform better in this context because they can capture sophisticated domain-specific information. Convolutional networks [12,13] and generative adversarial networks (GAN) [14,15,16] have shown to greatly improve the quality of the synthesized high-resolution audio. However, GANs are known to be hard to train and produce outputs with significant artifacts.…”

Section: Introductionmentioning

confidence: 99%

“…However, the LR audio encoder only provides a time-domain perspective and it is challenging for LR audio encoder to exploit several characteristics of the signal, including cyclic behavior and long-range dependence, due to the limited receptive field and model capacity. Therefore, we further introduce the frequency domain encoder called STFT encoder, which first applies short-time Fourier transform (STFT) to the waveform and then encodes the phase and magnitude spectrogram, both of which are widely used in modern speech applications and methods [16,24]. The contributions of this work are summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

WSRGlow: A Glow-Based Waveform Generative Model for Audio Super-Resolution

Zhang¹,

Ren

Xu³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

Audio super-resolution is the task of constructing a highresolution (HR) audio from a low-resolution (LR) audio by adding the missing band. Previous methods based on convolutional neural networks and mean squared error training objective have relatively low performance, while adversarial generative models are difficult to train and tune. Recently, normalizing flow has attracted a lot of attention for its high performance, simple training and fast inference. In this paper, we propose WSRGlow, a Glow-based waveform generative model to perform audio super-resolution. Specifically, 1) we integrate WaveNet and Glow to directly maximize the exact likelihood of the target HR audio conditioned on LR information; and 2) to exploit the audio information from low-resolution audio, we propose an LR audio encoder and an STFT encoder, which encode the LR information from the time domain and frequency domain respectively. The experimental results show that the proposed model is easier to train and outperforms the previous works in terms of both objective and perceptual quality. WSRGlow is also the first model to produce 48kHz waveforms from 12kHz LR audio. Audio samples are available at https://zkx06111.github.io/wsrglow/.

show abstract

BigWavGAN: A Wave-To-Wave Generative Adversarial Network for Music Super-Resolution

Zhang,

Watanabe

2023

2023 IEEE 12th Global Conference on Consumer Electronics (GCCE)

View full text Add to dashboard Cite

Phase-Aware Music Super-Resolution Using Generative Adversarial Networks

Cited by 17 publications

References 0 publications

BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networks

BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networks

WSRGlow: A Glow-Based Waveform Generative Model for Audio Super-Resolution

BigWavGAN: A Wave-To-Wave Generative Adversarial Network for Music Super-Resolution

Contact Info

Product

Resources

About