Revisiting the Onsets and Frames Model with Additive Attention

Cheuk, Kin Wai; Luo, Yin-Jyun; Benetos, Emmanouil; Herremans, Dorien

doi:10.1109/ijcnn52387.2021.9533407

Cited by 15 publications

(11 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a major advancement, U-net models [35] were shown to improve performance of AMT [3], [20]- [23], [36] and other MIR tasks [37], [38]. More recently, the inclusion of self-attention components into U-nets [3], [21] and other models [39] was applied successfully to MPE. Most of our architectures rely on the U-net paradigm, enhanced with self-attention components as in [3] and other extensions.…”

Section: A Previous Modelsmentioning

confidence: 99%

“…Multi-task strategies. While the models mentioned above are usually trained on the MPE task in isolation, major success was reported when proceeding to multi-task settings, most prominently the onset-and-frames approach [6], [7], [39]. However, this strategy is tailored to the piano-solo case (where the percussive keystrokes help to successfully track onsets) and therefore, is not generally applicable to other instruments with less prominent onsets.…”

Section: A Previous Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Weiß¹,

Peeters²

2022

Preprint

View full text Add to dashboard Cite

Extracting pitch information from music recordings is a challenging but important problem in music signal processing. Frame-wise transcription or multi-pitch estimation aims for detecting the simultaneous activity of pitches in polyphonic music recordings and has recently seen major improvements thanks to deep-learning techniques, with a variety of proposed network architectures. In this paper, we realize different architectures based on convolutional neural networks, the U-net structure, and self-attention components. We propose several modifications to these architectures including self-attention modules for skip connections, recurrent layers to replace the self-attention, and a multi-task strategy with simultaneous prediction of the degree of polyphony. We compare variants of these architectures in different sizes for multi-pitch estimation, focusing on Western classical music beyond the piano-solo scenario using the MusicNet and Schubert Winterreise datasets. Our experiments indicate that most architectures yield competitive results and that larger model variants seem to be beneficial. However, we find that these results substantially depend on randomization effects and the particular choice of the training-test split, which questions the claim of superiority for particular architectures given only small improvements. We therefore investigate the influence of dataset splits in the presence of several movements of a work cycle (cross-version evaluation) and propose a best-practice splitting strategy for MusicNet, which weakens the influence of individual test tracks and suppresses overfitting to specific works and recording conditions. A final evaluation on a mixed dataset suggests that improvements on one specific dataset do not necessarily generalize to other scenarios, thus emphasizing the need for further high-quality multi-pitch datasets in order to reliably measure progress in music transcription tasks.

show abstract

Section: A Previous Modelsmentioning

confidence: 99%

Section: A Previous Modelsmentioning

confidence: 99%

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Weiß¹,

Peeters²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Following existing literature [12,13,18,19,23,24], we report the frame-wise, note-wise, and note-with-offset-wise metrics to evaluate our model performance comprehensively. For note-wise metric, we use a onset tolerance of 50ms; for note-with-offset-wise metric, we use an offset tolerance of 50ms or 20% of the note duration, whichever is larger [2].…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…For note-wise metric, we use a onset tolerance of 50ms; for note-with-offset-wise metric, we use an offset tolerance of 50ms or 20% of the note duration, whichever is larger [2]. Readers are referred to Cheuk et al [13] which explains the differences between these metrics in detail in their Section IV-C. In our experiments, we use the implementations from mir_eval 1 to calculate and report the above-mentioned metrics.…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…Between the models that use and do not use onset inference, we can see that onset inference has the tendency of decreasing the frame-wise F1-score while improving the note-wise F1-scores. This is due to the unreliability of the frame-wise metric [13,18]. Cheuk et al [13] has provided a few examples and shown that a high framewise score does not guarantee a good transcription.…”

Section: One-shot Maps the Bottom Part Of Tablementioning

confidence: 99%

See 1 more Smart Citation

ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data

Cheuk

Herremans

2021

Proceedings of the 29th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Most of the current supervised automatic music transcription (AMT) models lack the ability to generalize. This means that they have trouble transcribing real-world music recordings from diverse musical genres that are not presented in the labelled training data. In this paper, we propose a semi-supervised framework, ReconVAT, which solves this issue by leveraging the huge amount of available unlabelled music recordings. The proposed ReconVAT uses reconstruction loss and virtual adversarial training. When combined with existing U-net models for AMT, ReconVAT achieves competitive results on common benchmark datasets such as MAPS and MusicNet. For example, in the few-shot setting for the string part version of MusicNet, ReconVAT achieves F1-scores of 61.0% and 41.6% for the note-wise and note-with-offset-wise metrics respectively, which translates into an improvement of 22.2% and 62.5% compared to the supervised baseline model. Our proposed framework also demonstrates the potential of continual learning on new data, which could be useful in real-world applications whereby new data is constantly available. CCS CONCEPTS• Applied computing → Sound and music computing; • Computing methodologies → Semi-supervised learning settings; Neural networks.

show abstract

Polyphonic Piano Music Transcription System Exploiting Mutual Correlations of Different Musical Note States

Kim,

Lee,

Kim

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Revisiting the Onsets and Frames Model with Additive Attention

Cited by 15 publications

References 6 publications

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

Deep-Learning Architectures for Multi-Pitch Estimation: Towards Reliable Evaluation

ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data

Polyphonic Piano Music Transcription System Exploiting Mutual Correlations of Different Musical Note States

Contact Info

Product

Resources

About