Automatic Generation of Piano Score Following Videos

Shan, Mengyi; Tsai, TJ

doi:10.5334/tismir.69

Cited by 3 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the same time, the composing process is also complex; therefore, music lovers cannot obtain a high number of network-required composition resources that makes composing more di cult. For this reason, this paper uses deep learning algorithm in the context of piano score recognition to solve this problem [3].…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Enabled Composition System Based on Piano Score Recognition

Shang

2022

Mobile Information Systems

View full text Add to dashboard Cite

Piano is used for music and comprises a stringed keyboard instrument wherein the strings are tapped by softer-coated wooden hammers. The score providing music for the piano, often a compressed transcription of orchestral music, is referred to as piano score. Presently, the Internet is overflowing with music score resources. Having so many music score resources available, professional learners and amateur music lovers are unable to identify and obtain music score information that matches their needs and wasting valuable time. Due to the rapid development of deep learning algorithms, some individuals utilize these algorithms to detect piano scores and construct composition systems, reducing the need of traditional machine learning algorithms on manual design and music knowledge guidelines. This paper uses the deep learning algorithm to construct piano score recognition framework based on K-Nearest Neighbor (KNN) algorithm and formulates the recognition system into multinote that significantly improves the recognition rate for the system. The self-attention mechanism is then introduced in order to build a composition system based on a deep learning algorithm in which composition training and processes are described. Finally, a comparative experiment is conducted to evaluate the recognition accuracy for the KNN-based piano score recognition system. The results show that highest recognition accuracy of this system is 67.5%. The effect of composition system is evaluated based on prediction accuracy of notes. Three experiments are conducted to train the composition notes. As a result, the prediction accuracy of experiments 1, 2, and 3 is 89.2%, 91.8%, and 92.7%, respectively, indicating that the system has a high prediction accuracy and a perfect composition effect.

show abstract

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Enabled Composition System Based on Piano Score Recognition

Shang

2022

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

Large-Scale Multimodal Piano Music Identification Using Marketplace Fingerprinting

Yang¹,

Goutam²,

Ji³

et al. 2022

Algorithms

View full text Add to dashboard Cite

This paper studies the problem of identifying piano music in various modalities using a single, unified approach called marketplace fingerprinting. The key defining characteristic of marketplace fingerprinting is choice: we consider a broad range of fingerprint designs based on a generalization of standard n-grams, and then select the fingerprint designs at runtime that are best for a specific query. We show that the large-scale retrieval problem can be framed as an economics problem in which a consumer and a store interact. In our analogy, the runtime search is like a consumer shopping in the store, the items for sale correspond to fingerprints, and purchasing an item corresponds to doing a fingerprint lookup in the database. Using basic principles of economics, we design an efficient marketplace in which the consumer has many options and adopts a rational buying strategy that explicitly considers the cost and expected utility of each item. We evaluate our marketplace fingerprinting approach on four different sheet music retrieval tasks involving sheet music images, MIDI files, and audio recordings. Using a database containing approximately 375,000 pages of sheet music, our method is able to achieve 0.91 mean reciprocal rank with sub-second average runtime on cell phone image queries. On all four retrieval tasks, the marketplace method substantially outperforms previous methods while simultaneously reducing average runtime. We present comprehensive experimental results, as well as detailed analyses to provide deeper intuition into system behavior.

show abstract

Piano Sheet Music Identification Using Marketplace Fingerprinting

Ji¹,

Yang²,

Tsai³

2021

View full text Add to dashboard Cite

Automatic Generation of Piano Score Following Videos

Cited by 3 publications

References 8 publications

A Deep Learning-Enabled Composition System Based on Piano Score Recognition

A Deep Learning-Enabled Composition System Based on Piano Score Recognition

Large-Scale Multimodal Piano Music Identification Using Marketplace Fingerprinting

Piano Sheet Music Identification Using Marketplace Fingerprinting

Contact Info

Product

Resources

About