MTD: A Multimodal Dataset of Musical Themes for MIR Research

Zalkow, Frank; Balke, Stefan; Arifi-Müller, Vlora; Müller, Meinard

doi:10.5334/tismir.68

Cited by 9 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process marks some temporal positions in the audio files (usually corresponding to beats or downbeats) that can be mapped to score positions. See, for example, the work on the Mazurka project (Sapp, 2007), the CrestMusePEDB dataset (Hashida et al, 2017), and the Musical Themes Dataset (Zalkow et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Automatic Note-Level Score-to-Performance Alignments in the ASAP Dataset

Peter¹,

Cancino-Chacón²,

Foscarin³

et al. 2023

Transactions of the International Society for Music Information Retrieval

View full text Add to dashboard Cite

Several MIR applications require fine-grained note alignments between MIDI performances and their musical scores for training and evaluation. However, large and high-quality datasets with this kind of data are not available, and their manual creation is a very time-consuming task that can only be performed by field experts. In this paper, we evaluate state-of-the-art automatic note alignment models applied to dataset generation. We increase the accuracy and reliability of the produced alignments with models that flexibly leverage existing annotations such as beat or measure alignments. We thoroughly evaluate these segment-constrained models and use the best to create note alignments for the ASAP dataset, a large dataset of solo piano MIDI performances beat-aligned to MusicXML scores. The resulting note alignments are manually checked and publicly available at: https://github.com/CPJKU/ asap-dataset. The contributions of this paper are four-fold: (1) we extend the ASAP dataset with reliable note alignments, thus creating (n)ASAP, the largest available fully note-aligned dataset, comprising more than 7 M annotated notes and close to 100 hours of music; (2) we design, evaluate, and publish segment-constrained models for note alignments that flexibly leverage existing annotations and significantly outperform automatic models; (3) we design, evaluate, and publish unconstrained automatic models for note alignment that produce results on par with the state of the art; (4) we introduce Parangonada, a web-interface for visualizing and correcting alignment annotations.

show abstract

Section: Related Workmentioning

confidence: 99%

Automatic Note-Level Score-to-Performance Alignments in the ASAP Dataset

Peter¹,

Cancino-Chacón²,

Foscarin³

et al. 2023

Transactions of the International Society for Music Information Retrieval

View full text Add to dashboard Cite

show abstract

“…Previous examples of visualizations in other interdisciplinary contexts may provide useful directions. For instance, Zalkow et al (2017) used chroma-based visualizations to explore the narrative in Wagner’s operas in collaboration with musicologists, and visualizations derived from MIR analyses have been incorporated in music education tools, such as Yousician. 3…”

Section: Various Use Cases Of Music Technology In Health Settingsmentioning

confidence: 99%

“… 2. Future of Music-IR Research Panel, 18th International Society for Music Information Retrieval Conference (ISMIR, 2017), Suzhou, China, https://ismir2017.ismir.net/panels/index.html …”

mentioning

confidence: 99%

Music, Computing, and Health: A Roadmap for the Current and Future Roles of Music Technology for Health Care and Well-Being

et al. 2021

Self Cite

View full text Add to dashboard Cite

The fields of music, health, and technology have seen significant interactions in recent years in developing music technology for health care and well-being. In an effort to strengthen the collaboration between the involved disciplines, the workshop “Music, Computing, and Health” was held to discuss best practices and state-of-the-art at the intersection of these areas with researchers from music psychology and neuroscience, music therapy, music information retrieval, music technology, medical technology (medtech), and robotics. Following the discussions at the workshop, this article provides an overview of the different methods of the involved disciplines and their potential contributions to developing music technology for health and well-being. Furthermore, the article summarizes the state of the art in music technology that can be applied in various health scenarios and provides a perspective on challenges and opportunities for developing music technology that (1) supports person-centered care and evidence-based treatments, and (2) contributes to developing standardized, large-scale research on music-based interventions in an interdisciplinary manner. The article provides a resource for those seeking to engage in interdisciplinary research using music-based computational methods to develop technology for health care, and aims to inspire future research directions by evaluating the state of the art with respect to the challenges facing each field.

show abstract

“…Automatic detection of motifs or themes also remains challenging, despite years of research [46]- [49]. The only public theme-related dataset, the Musical Theme Dataset [3], is only for Western classical music and does not specify all positions where the themes occur in compositions.…”

Section: Finding Thematic Conditioning Materialsmentioning

confidence: 99%

“…In Western classical music, both themes and motifs can be considered short, salient recurring patterns. Themes are musical ideas that convey a sense of "completeness" and "roundedness," and are essential elements used to build a composition, or a part of it[3]. In contrast, motifs are considered to be shorter and more basic[4].…”

mentioning

confidence: 99%

Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer

Shih¹,

Wu²,

Zalkow³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Attention-based Transformer models have been increasingly employed for automatic music generation. To condition the generation process of such a model with a user-specified sequence, a popular approach is to take that conditioning sequence as a priming sequence and ask a Transformer decoder to generate a continuation. However, this prompt-based conditioning cannot guarantee that the conditioning sequence would develop or even simply repeat itself in the generated continuation. In this paper, we propose an alternative conditioning approach, called theme-based conditioning, that explicitly trains the Transformer to treat the conditioning sequence as a thematic material that has to manifest itself multiple times in its generation result. This is achieved with two main technical contributions. First, we propose a deep learning-based approach that uses contrastive representation learning and clustering to automatically retrieve thematic materials from music pieces in the training data. Second, we propose a novel gated parallel attention module to be used in a sequence-to-sequence (seq2seq) encoder/decoder architecture to more effectively account for a given conditioning thematic material in the generation process of the Transformer decoder. We report on objective and subjective evaluations of variants of the proposed Theme Transformer and the conventional promptbased baseline, showing that our best model can generate, to some extent, polyphonic pop piano music with repetition and plausible variations of a given condition.

show abstract

MTD: A Multimodal Dataset of Musical Themes for MIR Research

Cited by 9 publications

References 28 publications

Automatic Note-Level Score-to-Performance Alignments in the ASAP Dataset

Automatic Note-Level Score-to-Performance Alignments in the ASAP Dataset

Music, Computing, and Health: A Roadmap for the Current and Future Roles of Music Technology for Health Care and Well-Being

Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer

Contact Info

Product

Resources

About