On Loss Functions and Evaluation Metrics for Music Source Separation

Enric, Gusó,; Pons, Jordi; Pascual, Santiago; Serrà, Joan

doi:10.1109/icassp43922.2022.9746530

Cited by 8 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where Y s denotes the ground-truth magnitude spectrogram of a target source. For an investigation of various loss functions used with the UMX network, we refer to [46]. As indicated in Table I, UMX is the model with fewest parameters among different approaches.…”

Section: B Open-unmix (Umx)mentioning

confidence: 99%

Source Separation of Piano Concertos Using Musically Motivated Augmentation Techniques

Özer,

Müller

2024

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

In this work, we address the novel and rarely considered source separation task of decomposing piano concerto recordings into separate piano and orchestral tracks. Being a genre written for a pianist typically accompanied by an ensemble or orchestra, piano concertos often involve an intricate interplay of the piano and the entire orchestra, leading to high spectrotemporal correlations between the constituent instruments. Moreover, in the case of piano concertos, the lack of multi-track data for training constitutes another challenge in view of datadriven source separation approaches. As a basis for our work, we adapt existing deep learning (DL) techniques, mainly used for the separation of popular music recordings. In particular, we investigate spectrogram-and waveform-based approaches as well as hybrid models operating in both spectrogram and waveform domains. As a main contribution, we introduce a musically motivated data augmentation approach for training based on artificially generated samples. Furthermore, we systematically investigate the effects of various augmentation techniques for DLbased models. For our experiments, we use a recently published, open-source dataset of multi-track piano concerto recordings. Our main findings demonstrate that the best source separation performance is achieved by a hybrid model when combining all augmentation techniques.

show abstract

Section: B Open-unmix (Umx)mentioning

confidence: 99%

Source Separation of Piano Concertos Using Musically Motivated Augmentation Techniques

Özer,

Müller

2024

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…In this setting, a mixture is fed to a parametric model (i.e., a neural network) that outputs the separated sources. Training is typically performed in a supervised manner by matching the estimated separations with the ground truth sources with a regression loss (e.g., L 1 or L 2 ) (Gusó et al 2022). Supervised regression has been applied to image source separation (Halperin, Ephrat, and Hoshen 2019), but it has been mainly investigated in the audio domain, where two approaches are prevalent: the mask-based approach and the waveform approach.…”

Section: Regression-based Source Separationmentioning

confidence: 99%

Latent Autoregressive Source Separation

Emilian

Mariani

Michele

et al. 2023

AAAI

View full text Add to dashboard Cite

Autoregressive models have achieved impressive results over a wide range of domains in terms of generation quality and downstream task performance. In the continuous domain, a key factor behind this success is the usage of quantized latent spaces (e.g., obtained via VQ-VAE autoencoders), which allow for dimensionality reduction and faster inference times. However, using existing pre-trained models to perform new non-trivial tasks is difficult since it requires additional fine-tuning or extensive training to elicit prompting. This paper introduces LASS as a way to perform vector-quantized Latent Autoregressive Source Separation (i.e., de-mixing an input signal into its constituent sources) without requiring additional gradient-based optimization or modifications of existing models. Our separation method relies on the Bayesian formulation in which the autoregressive models are the priors, and a discrete (non-parametric) likelihood function is constructed by performing frequency counts over latent sums of addend tokens. We test our method on images and audio with several sampling strategies (e.g., ancestral, beam search) showing competitive results with existing approaches in terms of separation quality while offering at the same time significant speedups in terms of inference time and scalability to higher dimensional data.

show abstract

“…For example, a recent paper found that while their low-rate speech coder per- formed worse according to POLQA [270], a subjective evaluation showed that their coder had similar performance to competing high-rate coders [271]. More generally, objective measures are known to not correlate well with subjective quality [28,29]. Hence we used a subjective test for the evaluation of our speech enhancement system.…”

Section: Subjective Evaluationmentioning

confidence: 99%

“…The SI-SNR is also used to evaluate and compare these models. Importantly, [28,29] recently showed that the SI-SNR is not well correlated with perceptual quality for speech separation systems. Hence, the field is potentially comparing models using a measure not indicative of perceptual quality.…”

Section: Investigation Of the Relationship Between Si-snr And Percept...mentioning

confidence: 99%

See 1 more Smart Citation

Improving Variational Methods for Representation Learning and Generative Modelling of Speech

Braithwaite¹

View full text Add to dashboard Cite

Hearing aids are clinically proven to improve the quality of life of hearing-impaired individuals. However, despite this, many people who could benefit from a hearing aid do not use one. The most common reason for this is that they perform poorly in noisy environments. Machine learning presents a collection of powerful methods for attenuating noise in speech signals. Moreover, state-of-the-art hearing aids are being designed with more powerful processors that can run machine learning algorithms. Several key factors make the deployment of machine learning to hearing aids challenging. Firstly, many state-of-the-art machine learning methods have high resource requirements (e.g., GPUs or large amounts of memory). Hence, these resource-intensive models cannot be run in real-time in low-complexity systems. Secondly, methods need to be causal (i.e., do not depend on access to future information). More generally, an issue is that the objectives that speech enhancement/separation models typically optimise (e.g., L1 or L2) correlate poorly with perceptual quality. The goal of this thesis is to develop improved models for attenuating noise in speech signals. Specifically, we aim to develop more principled models and better understand existing frameworks. Overall, the objective is to develop low-complexity models for attenuating noise in speech signals. We begin by developing an improved autoencoder-based generative model, which we name the Bounded Information Rate Variational Autoencoder. This model resolves the posterior collapse issue with Variational Autoencoders by pre-specifying the information rate between the data and latent variables. In a further extension of this model, we relax the constraints on the distribution of latent variables. Hence, the variance of this distribution is constrained, but not the shape. We name this model the Variance Constrained Autoencoder (VCAE). The VCAE outperforms other state-of-the-art generative models in both reconstruction and generation quality. In subsequent work, we applied the Variance Constrained Autoencoder to speech enhancement. The model is trained to reconstruct a clean target signal from the noisy input. Importantly, rather than enhancing large segments, we enhanced short blocks ($\tilde 4$ ms). This allowed our system to have a lower computation complexity and higher subjective performance than other state-of-the-art approaches. Our final contribution is an empirical study of a speech separation system called Conv-TasNet. This model separates speech signals by applying multiplicative masks to the mixture signal in a learned over-parameterised transform domain. The phenomenon of interest is the structure of the learned transform. Specifically, an over-parameterised frequency domain transform is learned, where more emphasis is placed on the regions with more signal energy. Hence, the transformation resembles something that is perceptually motivated. In the empirical study of Conv-TasNet, we first provide empirical evidence that shows the transform learned by Conv-TasNet prefers uncorrelated coefficients. This explains why a frequency domain transform is learned. Moreover, we show that performance can be improved over the base system by reducing the correlation of the transform space coefficients. In the second part of this contribution, we demonstrate a link between the flatness of the filterbank and the overall separation performance. Furthermore, we show that the density of the vectors affects the flatness of the frame. In other words, we provide evidence that the perceptual structure of the learned filterbank results from an implicit bias in stochastic gradient descent training.

show abstract

On Loss Functions and Evaluation Metrics for Music Source Separation

Cited by 8 publications

References 25 publications

Source Separation of Piano Concertos Using Musically Motivated Augmentation Techniques

Source Separation of Piano Concertos Using Musically Motivated Augmentation Techniques

Latent Autoregressive Source Separation

Improving Variational Methods for Representation Learning and Generative Modelling of Speech

Contact Info

Product

Resources

About