Piano Sustain-pedal Detection Using Convolutional Neural Networks

Liang, Beici; Fazekas, György; Sandler, Mark

doi:10.1109/icassp.2019.8683505

Cited by 7 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relation between expressive features and emotions has been studied with regard to vibratos and articulations [121], [122]. Methods to characterise expressive techniques have recently been proposed such as detection of vibrato in violin and erhu [108], [123], arpeggios in multiple instruments [107], use of pedal in piano [124], and representative playing techniques in guitar [125], [126] and bamboo flute [127]. These embellishments are unique to individual performances and computational models would be useful for comparison 14.…”

Section: Connecting Emergent Perceptual Themes and Mirmentioning

confidence: 99%

Examining Emotion Perception Agreement in Live Music Performance

Yang

Reed

Chew

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Current music emotion recognition (MER) systems rely on emotion data averaged across listeners and over time to infer the emotion expressed by a musical piece, often neglecting time-and listener-dependent factors. These limitations can restrict the efficacy of MER systems and cause misjudgements. We present two exploratory studies on music emotion perception. First, in a live music concert setting, fifteen audience members annotated perceived emotion in the valence-arousal space over time using a mobile application. Analyses of inter-rater reliability yielded widely varying levels of agreement in the perceived emotions. A follow-up lab-based study to uncover the reasons for such variability was conducted, where twenty-one participants annotated their perceived emotions whilst viewing and listening to a video recording of the original performance and offered open-ended explanations. Thematic analysis revealed salient features and interpretations that help describe the cognitive processes underlying music emotion perception. Some of the results confirm known findings of music perception and MER studies. Novel findings highlight the importance of less frequently discussed musical attributes, such as musical structure, performer expression, and stage setting, as perceived across audio and visual modalities. Musicians are found to attribute emotion change to musical harmony, structure, and performance technique more than non-musicians. We suggest that accounting for such listener-informed music features can benefit MER in helping to address variability in emotion perception by providing reasons for listener similarities and idiosyncrasies.

show abstract

Section: Connecting Emergent Perceptual Themes and Mirmentioning

confidence: 99%

Examining Emotion Perception Agreement in Live Music Performance

Yang

Reed

Chew

et al. 2023

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

“…Extraction of the LMS L of the audio content related to the video sequence. Indeed, the LMS represents a very informative tool for audio data and was used several times as a valuable feature for audio and speech classification and processing [24,[37][38][39][40][41].…”

Section: Figurementioning

confidence: 99%

CNN-Based Multi-Modal Camera Model Identification on Video Sequences

et al. 2021

View full text Add to dashboard Cite

Identifying the source camera of images and videos has gained significant importance in multimedia forensics. It allows tracing back data to their creator, thus enabling to solve copyright infringement cases and expose the authors of hideous crimes. In this paper, we focus on the problem of camera model identification for video sequences, that is, given a video under analysis, detecting the camera model used for its acquisition. To this purpose, we develop two different CNN-based camera model identification methods, working in a novel multi-modal scenario. Differently from mono-modal methods, which use only the visual or audio information from the investigated video to tackle the identification task, the proposed multi-modal methods jointly exploit audio and visual information. We test our proposed methodologies on the well-known Vision dataset, which collects almost 2000 video sequences belonging to different devices. Experiments are performed, considering native videos directly acquired by their acquisition devices and videos uploaded on social media platforms, such as YouTube and WhatsApp. The achieved results show that the proposed multi-modal approaches significantly outperform their mono-modal counterparts, representing a valuable strategy for the tackled problem and opening future research to even more challenging scenarios.

show abstract

“…All notes played will continue to sound until the pedal is released. However, many previous piano transcription systems [24], [15], [30] did not incorporate sustain pedal transcription into their piano transcription systems, and sustain-pedal transcription systems [31] did not transcribe piano notes. In [31], a convolutional neural network was used to detect piano pedals in frame-wise.…”

Section: Sustain Pedal Transcriptionmentioning

confidence: 99%

“…However, many previous piano transcription systems [24], [15], [30] did not incorporate sustain pedal transcription into their piano transcription systems, and sustain-pedal transcription systems [31] did not transcribe piano notes. In [31], a convolutional neural network was used to detect piano pedals in frame-wise. However, there is a lack of benchmark sustain pedal transcription system on the MAESTRO dataset [25].…”

Section: Sustain Pedal Transcriptionmentioning

confidence: 99%

“…Furthermore, previous piano transcription systems [24], [25] do not contain sustain pedal transcription, which is an important part of piano transcription, and are frequently used by pianists. Liang et al applied convolutional neural networks to detect sustain pedals [31], [32]. However, there is a lack of research on building piano transcription systems with both note and sustain pedal transcription.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times

Kong¹,

Li²,

Song³

et al. 2020

Preprint

View full text Add to dashboard Cite

Automatic music transcription (AMT) is the task of transcribing audio recordings into symbolic representations such as Musical Instrument Digital Interface (MIDI). Recently, neural networks based methods have been applied to AMT, and have achieved state-of-the-art result. However, most of previous AMT systems predict the presence or absence of notes in the frames of audio recordings. The transcription resolution of those systems are limited to the hop size time between adjacent frames. In addition, previous AMT systems are sensitive to the misaligned onsets and offsets labels of audio recordings. For high-resolution evaluation, previous works have not investigated AMT systems evaluated with different onsets and offsets tolerances. For piano transcription, there is a lack of research on building AMT systems with both note and pedal transcription. In this article, we propose a high-resolution AMT system trained by regressing precise times of onsets and offsets. In inference, we propose an algorithm to analytically calculate the precise onsets and offsets times of note and pedal events. We build both note and pedal transcription systems with our high-resolution AMT system. We show that our AMT system is robust to misaligned onsets and offsets labels compared to previous systems. Our proposed system achieves an onset F1 of 96.72% on the MAESTRO dataset, outperforming the onsets and frames system from Google of 94.80%. Our system achieves a pedal onset F1 score of 91.86%, and is the first benchmark result on the MAESTRO dataset. We release the source code of our work at https://github.com/bytedance/piano_transcription.

show abstract

Piano Sustain-pedal Detection Using Convolutional Neural Networks

Cited by 7 publications

References 10 publications

Examining Emotion Perception Agreement in Live Music Performance

Examining Emotion Perception Agreement in Live Music Performance

CNN-Based Multi-Modal Camera Model Identification on Video Sequences

High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times

Contact Info

Product

Resources

About