Interactive multimodal music transcription

Iñesta, José M.; Pérez-Sancho, Carlos

doi:10.1109/icassp.2013.6637639

Cited by 4 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under this premise some works have considered the use of onset information to address such issues. Examples of such works may be found in Marolt and Divjak (2002), which considered onset information for tackling the problem of tracking repeated notes, the work by Emiya, Badeau, and David (2008), in which onset information was used for segmenting the signal before the pitch estimation phase, the proposal by Iñesta and Pérez-Sancho (2013), which postprocessed the result of the MPE stage with the aim of correcting timing issues with onset information, or the system by Grosche et al (2012), which also considered onset information under an HMM framework. Note that, while scarce, some works as the one by Benetos and Dixon (2011) have considered both onset and offset estimation systems for tackling these timing issues.…”

Section: Background On Note Trackingmentioning

confidence: 99%

“…On the contrary, note tracking has not received that much attention despite its relevance in the overall success of the automatic music transcription task (Duan & Temperley, 2014). Note-level transcriptions are commonly obtained by binarizing the time-pitch representation and post-processing it with a set of minimum-length pruning processes for eliminating spurious detections and gap-filling stages for removing small gaps between consecutive pitches as, for instance, works by Benetos and Weyde (2015) or Iñesta and Pérez-Sancho (2013). Thus, our main criticism lies in the fact that note tracking strategies typically rely on hand-crafted rules.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A supervised classification approach for note tracking in polyphonic piano transcription

Valero-Mas

Benetos

Iñesta

2018

Journal of New Music Research

View full text Add to dashboard Cite

In the field of Automatic Music Transcription, note tracking systems constitute a key process in the overall success of the task as they compute the expected notelevel abstraction out of a frame-based pitch activation representation. Despite its relevance, note tracking is most commonly performed using a set of hand-crafted rules adjusted in a manual fashion for the data at issue. In this regard the present work introduces an approach based on machine learning, and more precisely supervised classification, that aims at automatically inferring such policies for the case of piano music. The idea is to segment each pitch band of a frame-based pitch activation into single instances which are subsequently classified as active or non-active note events. Results using a comprehensive set of supervised classification strategies on the MAPS piano dataset report its competitiveness against other commonly considered strategies for note tracking as well as an improvement of more than +10 % in terms of F-measure when compared to the baseline considered for both frame-level and notelevel evaluation.

show abstract

Section: Background On Note Trackingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A supervised classification approach for note tracking in polyphonic piano transcription

Valero-Mas

Benetos

Iñesta

2018

Journal of New Music Research

View full text Add to dashboard Cite

show abstract

“…Moreover, the interactive-predictive approach can be generalized to applications beyond IMT. The framework is applicable to the transcription of handwritten text documents (Toselli et al, 2007;Martín-Albo et al, 2013), ancient manuscripts (Granell et al, 2016), music (Inesta and Pérez-Sancho, 2013), layout detection (Quirós et al, 2017), parsing (Toselli et al, 2011), speech transcription , image retrieval (Segarra et al, 2011) or multimodal captioning (Peris and Casacuberta, 2019b).…”

Section: Multimodal Interactionmentioning

confidence: 99%

Interactivity, Adaptation and Multimodality in Neural Sequence-to-sequence Learning

Abril¹

View full text Add to dashboard Cite

Chapter 1 frames the scope of this thesis, introducing the pattern recognition field and, more specifically, the MT field. It reviews the different historical approaches devised to tackle this problem. Moreover, it sets the experimental framework followed in this thesis and the main scientific objectives. Chapter 2 describes the mathematical model that represents the core of the thesis: neural networks. It addresses the parameter estimation process, describes different neural architectures and a number of techniques used along the thesis to improve the generalization capability of the model. Chapter 3 introduces the neural machine translation technology, describing the most common architectures and decoding process. Moreover, it reviews different aspects relating the NMT field that nowadays receive the attention of the research community. It also compares NMT in the different translation tasks that will be tackled in the thesis. Chapter 4 introduces the interactive-predictive pattern recognition field, that aims to minimize the effort spent by the user while supervising an automatic system. It proposes the application of this theoretical framework to the neural technology, introducing alternative interaction protocols. After that, these interactivepredictive neural systems are evaluated. Chapter 5 describes the adaptation of NMT systems via online learning techniques. After receiving a corrected sample, the system can be updated to include this new knowledge. Here are described the methods to perform this adaptation and introduces two novel alternatives. In addition, an active learning framework for neural systems is proposed, useful for a situation that requires the translation of large amounts of data. All these scenarios are thoroughly evaluated in a variety of conditions, including a user evaluation involving professional post-editors. Chapter 6 departs from the MT problem to tackle different multimodal sequenceto-sequence tasks. More precisely, it is focused on the generation textual descriptions of videos. These techniques are also applied to the captioning of daily events, captured with an egocentric camera. Finally, the interactive-predictive framework described in Chapter 4 is applied to these multimodal systems. Chapter 7 draws the main conclusions of the thesis, describing the scientific contributions and publications derived from it and traces several lines of future research. These chapters are complemented by two appendices. Appendix A describes NMT-Keras, an open-source library developed to build neural models, that has been used to carry out most of the experiments described in the thesis. In Appendix B we provide the results of a survey carried out in the scope of Chapter 5.

show abstract