Polyphonic pitch tracking with deep layered learning

Elowsson, Anders

doi:10.1121/10.0001468

Cited by 20 publications

(11 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We finally want to stress again that all these models perform well, most of them superior to previously reported results: All models outperform our previous results [30] with a smaller network similar to CNN:XS, especially for SWD. For B10, our best Unet:XL reaches Acc=87.2% as compared to a traditional method [13] with Acc<70% and a deep-learning approach [48] with up to Acc=85.6%. On PhA, DRCNN:L and Unet:XL reach up to AP=87.2%, as compared to our own MCTC-based results of AP=82.8% [30].…”

Section: E Evaluation With a Mixed Datasetmentioning

confidence: 94%

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Weiß¹,

Peeters²

2022

Preprint

View full text Add to dashboard Cite

Extracting pitch information from music recordings is a challenging but important problem in music signal processing. Frame-wise transcription or multi-pitch estimation aims for detecting the simultaneous activity of pitches in polyphonic music recordings and has recently seen major improvements thanks to deep-learning techniques, with a variety of proposed network architectures. In this paper, we realize different architectures based on convolutional neural networks, the U-net structure, and self-attention components. We propose several modifications to these architectures including self-attention modules for skip connections, recurrent layers to replace the self-attention, and a multi-task strategy with simultaneous prediction of the degree of polyphony. We compare variants of these architectures in different sizes for multi-pitch estimation, focusing on Western classical music beyond the piano-solo scenario using the MusicNet and Schubert Winterreise datasets. Our experiments indicate that most architectures yield competitive results and that larger model variants seem to be beneficial. However, we find that these results substantially depend on randomization effects and the particular choice of the training-test split, which questions the claim of superiority for particular architectures given only small improvements. We therefore investigate the influence of dataset splits in the presence of several movements of a work cycle (cross-version evaluation) and propose a best-practice splitting strategy for MusicNet, which weakens the influence of individual test tracks and suppresses overfitting to specific works and recording conditions. A final evaluation on a mixed dataset suggests that improvements on one specific dataset do not necessarily generalize to other scenarios, thus emphasizing the need for further high-quality multi-pitch datasets in order to reliably measure progress in music transcription tasks.

show abstract

Section: E Evaluation With a Mixed Datasetmentioning

confidence: 94%

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Weiß¹,

Peeters²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In a very thorough domain-specific treatment, Elowsson [23] constructs a hierarchical model that extracts fundamental frequency contours from spectrograms and uses these contours to infer note onsets and offsets. While it can be useful for many applications to have such intermediate representations as in Engel et al [24], in this work we treat polyphonic transcription from audio to discrete notes as an end-to-end problem.…”

Section: Related Work 21 Piano Transcriptionmentioning

confidence: 99%

Sequence-to-Sequence Piano Transcription with Transformers

Hawthorne¹,

Simon²,

Swavely³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automatic Music Transcription has seen significant progress in recent years by training custom deep neural networks on large datasets. However, these models have required extensive domain-specific design of network architectures, input/output representations, and complex decoding schemes. In this work, we show that equivalent performance can be achieved using a generic encoderdecoder Transformer with standard decoding methods. We demonstrate that the model can learn to translate spectrogram inputs directly to MIDI-like output events for several transcription tasks. This sequence-to-sequence approach simplifies transcription by jointly modeling audio features and language-like output dependencies, thus removing the need for task-specific architectures. These results point toward possibilities for creating new Music Information Retrieval models by focusing on dataset creation and labeling rather than custom model design. Recent progress in piano transcription has been largely driven by two factors: the construction and release of datasets containing aligned piano audio and MIDI (most notably MAPS [1] and MAESTRO [2]) and the use of deep neural networks with architectures specifically designed for piano transcription (e.g., the Onsets and Frames architecture that models note onsets and note presence separately [3]). While domain-specific models have lead to improvements on benchmark datasets, it is not clear if these Work done as a Google Brain Student Researcher.

show abstract

“…Though varied, much of the cutting-edge research is reliant on machine learning (ML) techniques, not necessarily seeking to better understand the underlying structures present, and opting rather to maximize efficacy of the respective approaches (Figure 1). Elowsson (2020) proposed a method for MPE that relies on "deep layered learning" (Elowsson and Friberg 2014;Elowsson 2018). It uses a multi-stage system of neural networks and processing steps to elicit pitch contours -i.e.…”

Section: Related Workmentioning

confidence: 99%

A geometric framework for pitch estimation on acoustic musical signals

Goodman

Gemst

Tiňo

2021

Journal of Mathematics and Music

View full text Add to dashboard Cite

This paper presents a geometric approach to pitch estimation (PE) -an important problem in music information retrieval (MIR), and a precursor to a variety of other problems in the field. Though there exist a number of highly accurate methods, both mono-pitch estimation and multi-pitch estimation (particularly with unspecified polyphonic timbre) prove computationally and conceptually challenging. A number of current techniques, while incredibly effective, are not targeted towards eliciting the underlying mathematical structures that underpin the complex musical patterns exhibited by acoustic musical signals. Tackling the approach from both theoretical and experimental perspectives, we present a novel framework, a basis for further work in the area, and results that (while not state of the art) demonstrate relative efficacy. The framework presented in this paper opens up a completely new way to tackle PE problems and may have uses both in traditional analytical approaches as well as in the emerging machine learning (ML) methods that currently dominate the literature.

show abstract

Polyphonic pitch tracking with deep layered learning

Cited by 20 publications

References 16 publications

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Sequence-to-Sequence Piano Transcription with Transformers

A geometric framework for pitch estimation on acoustic musical signals

Contact Info

Product

Resources

About