Perceptual loss function for neural modeling of audio systems

Wright, Alec; Välimäki, Vesa

doi:10.1109/icassp40776.2020.9052944

Cited by 17 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The tests in Section 2 and in previous work [21,23] used a first-order high-pass pre-emphasis filter. For the experiments presented in this section we use a perceptually motivated pre-emphasis filter based on a low-passed A-Weighting filter, which was first proposed in and tested in [37]. The frequency response of this pre-emphasis filter is shown in Figure 11.…”

Section: Loss Function and Trainingmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Guitar Amplifier Emulation with Deep Learning

Wright

Damskägg²,

Juvela³

et al. 2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

This article investigates the use of deep neural networks for black-box modelling of audio distortion circuits, such as guitar amplifiers and distortion pedals. Both a feedforward network, based on the WaveNet model, and a recurrent neural network model are compared. To determine a suitable hyperparameter configuration for the WaveNet, models of three popular audio distortion pedals were created: the Ibanez Tube Screamer, the Boss DS-1, and the Electro-Harmonix Big Muff Pi. It is also shown that three minutes of audio data is sufficient for training the neural network models. Real-time implementations of the neural networks were used to measure their computational load. To further validate the results, models of two valve amplifiers, the Blackstar HT-5 Metal and the Mesa Boogie 5:50 Plus, were created, and subjective tests were conducted. The listening test results show that the models of the first amplifier could be identified as different from the reference, but the sound quality of the best models was judged to be excellent. In the case of the second guitar amplifier, many listeners were unable to hear the difference between the reference signal and the signals produced with the two largest neural network models. This study demonstrates that the neural network models can convincingly emulate highly nonlinear audio distortion circuits, whilst running in real-time, with some models requiring only a relatively small amount of processing power to run on a modern desktop computer.

show abstract

Section: Loss Function and Trainingmentioning

confidence: 99%

“…As proposed in [21] and used in [37], an additional loss function representing the difference in DC offset between the target and neural network output was also included:…”

Section: Loss Function and Trainingmentioning

confidence: 99%

Real-Time Guitar Amplifier Emulation with Deep Learning

Wright

Damskägg²,

Juvela³

et al. 2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use weight normalization [51], which we found to improve results and training stability. At the end of the trainable blocks, we add a fixed pre-emphasis (PE) filter with impulse response [1, −0.97], which amplifies the high frequencies, as common in similar tasks [58,59]. An illustration of the generator's architecture is shown in Fig.…”

Section: Methodsmentioning

confidence: 99%

Catch-A-Waveform: Learning to Generate Audio from a Single Short Example

Greshler¹,

Shaham²,

Michaeli³

2021

Preprint

View full text Add to dashboard Cite

Models for audio generation are typically trained on hours of recordings. Here, we illustrate that capturing the essence of an audio source is typically possible from as little as a few tens of seconds from a single training signal. Specifically, we present a GAN-based generative model that can be trained on one short audio signal from any domain (e.g. speech, music, etc.) and does not require pre-training or any other form of external supervision. Once trained, our model can generate random samples of arbitrary duration that maintain semantic similarity to the training waveform, yet exhibit new compositions of its audio primitives. This enables a long line of interesting applications, including generating new jazz improvisations or new a-cappella rap variants based on a single short example, producing coherent modifications to famous songs (e.g. adding a new verse to a Beatles song based solely on the original recording), filling-in of missing parts (inpainting), extending the bandwidth of a speech signal (super-resolution), and enhancing old recordings without access to any clean training example. We show that in all cases, no more than 20 seconds of training audio commonly suffice for our model to achieve state-of-the-art results. This is despite its complete lack of prior knowledge about the nature of audio signals in general.Preprint. Under review.

show abstract

“…Various deep learning approaches have already been proposed for the task of modeling audio effects [12][13][14][15][16][17]. While previous approaches have focused on training a single model for each effect, we believe our work is the first to consider building a model that emulates a series connection of effects and their parameters, jointly.…”

Section: Transformation Networkmentioning

confidence: 99%

Automatic Multitrack Mixing With A Differentiable Mixing Console Of Neural Audio Effects

Steinmetz

Pons

Pascual

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relatively unstructured and variable. To address these challenges, we propose a domain-inspired model with a strong inductive bias for the mixing task. We achieve this with the application of pre-trained sub-networks and weight sharing, as well as with a sum/difference stereo loss function. The proposed model can be trained with a limited number of examples, is permutation invariant with respect to the input ordering, and places no limit on the number of input sources. Furthermore, it produces human-readable mixing parameters, allowing users to manually adjust or refine the generated mix. Results from a perceptual evaluation involving audio engineers indicate that our approach generates mixes that outperform baseline approaches. To the best of our knowledge, this work demonstrates the first approach in learning multitrack mixing conventions from real-world data at the waveform level, without knowledge of the underlying mixing parameters.

show abstract

Perceptual loss function for neural modeling of audio systems

Cited by 17 publications

References 12 publications

Real-Time Guitar Amplifier Emulation with Deep Learning

Real-Time Guitar Amplifier Emulation with Deep Learning

Catch-A-Waveform: Learning to Generate Audio from a Single Short Example

Automatic Multitrack Mixing With A Differentiable Mixing Console Of Neural Audio Effects

Contact Info

Product

Resources

About