A Deep Approach for Handwritten Musical Symbols Recognition

Pereira, Roberto Matheus Pinheiro; Matos, Caio; Bráz, Geraldo; Almeida, João Dallyson Sousa de; Paiva, Anselmo Cardoso de

doi:10.1145/2976796.2988171

Cited by 8 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Experimental results showed that even without post-processing, this approach outperformed many traditional methods. Pinheiro et al [37] used ConvNets for note recognition and compared it with classic networks like LeNet, AlexNet, and GoogleNet, with GoogleNet showing the best performance in note recognition. Rebelo et al [18] were the first to introduce neural networks in the note classification stage, laying the foundation for further research.…”

Section: B Deep Learning-based Methodsmentioning

confidence: 99%

Deep Multilevel Cascade Residual Recurrent Framework (MCRR) for Sheet Music Recognition

Yu,

Chen

2024

IEEE Access

View full text Add to dashboard Cite

Sheet music recognition is a vital technology aimed at converting printed or handwritten musical scores into digital or machine-readable formats. The significance of this technology lies in making music compositions more accessible for editing, performance, learning, and sharing, thereby fostering music education, composition, and culture. It also provides a powerful tool for music analysis, research, and preservation. Our aim is to investigate a sheet music recognition method that offers a simple workflow, high recognition accuracy, and fast model convergence. Specifically, the proposed Deep Multilevel Cascade Residual Recurrent (MCRR) framework for sheet music recognition consists of the following components. Firstly, we introduce additive Gaussian white noise, additive Perlin noise, and elastic deformations such as rotation and stretching to simulate real-world noise in the sheet music images, thereby augmenting the dataset, enhancing model robustness, and mitigating overfitting. Secondly, in the feature extraction phase, we employ a residual Convolutional Neural Network (ConvNet) to address the issue of model degradation and use the multilevel cascade fusion technique to obtain comprehensive feature information, improving the model's feature extraction capability and reducing recognition errors. For note recognition, we use a variant of RNN (Recurrent Neural Network) called SRU (Simple Recurrent Unit), which transforms most computations into parallel processing, speeding up model convergence. Finally, we combine the Connectionist Temporal Classification (CTC) loss function with SRU to eliminate the requirement for strict alignment between data and labels, enabling note classification and recognition. Extensive ablation experiments and comparative analyses, including visual analysis, intuitive illustrations, and quantitative assessments, confirm the effectiveness of the proposed method, demonstrating its superiority over various state-of-the-art methods. The proposed method achieved promising results in both the PrIMus and Camera-PrIMuS datasets. Specifically, in the PrIMus dataset, the method obtained an SeER (Symbol Error Rate) of 1.4571% and a SyER (System Error Rate) of 0.3234%. Notably, it demonstrated high accuracy in pitch, type, and note recognition, scoring approximately 97% in pitch and type accuracy and around 94% in note accuracy. The training time per epoch was relatively low, recorded at 0.56 seconds. In the case of the Camera-PrIMuS dataset, the method achieved slightly lower but still competitive results. It exhibited an SeER of 5.1488% and a SyER of 1.0612%, with pitch and type accuracies around 90%, and note accuracy at approximately 88%. The training time per epoch was slightly higher at 1.93 seconds Furthermore, we compare our method with existing commercial software, namely Capella-scan, PhotoScore, and SmartScore. Among these, Capella-scan delivers the best performance but exhibits lower robustness compared to the proposed method.INDEX TERMS Sheet music recognition, deep learning, multile...

show abstract

Section: B Deep Learning-based Methodsmentioning

confidence: 99%

Deep Multilevel Cascade Residual Recurrent Framework (MCRR) for Sheet Music Recognition

Yu,

Chen

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This makes most research that uses custom CNN architecture have difficulty beating the OMR research that uses official CNN architecture. An example can be seen from the previously mentioned research where the research with the HOMUS dataset gets 96.01% accuracy by using GoogleNet [11] which is bigger than the one that uses custom architecture which gets 95.56% accuracy [21]. It is proven more with the research that uses a lesser amount of data and classes but has quite low accuracy which is 80% [22] if compared to the research that uses GoogleNet [11].…”

Section: Introductionmentioning

confidence: 98%

“…An example can be seen from the previously mentioned research where the research with the HOMUS dataset gets 96.01% accuracy by using GoogleNet [11] which is bigger than the one that uses custom architecture which gets 95.56% accuracy [21]. It is proven more with the research that uses a lesser amount of data and classes but has quite low accuracy which is 80% [22] if compared to the research that uses GoogleNet [11]. Based on this information, it is better to try the currently developed CNN architecture that has been available and has proven good in another research field.…”

Section: Introductionmentioning

confidence: 99%

Recognition of music symbol notation using convolutional neural network

Setyo,

Kusuma

2024

IJECE

View full text Add to dashboard Cite

Musical notation is one thing that needs to be learned to play music. This notation has an important role in music because it can help in visualizing instructions for playing musical instruments and singing. Unfortunately, musical symbols that are commonly written in musical notation are difficult for beginners who have just started learning music. This research proposed a solution to create an optical music recognition (OMR) using a deep learning model to classify musical notes more accurately with some of the latest convolutional neural network (CNN) architectures. The research was carried out by implementing vision transformer (ViT), CoAtNet-0, and ConvNeXt-Tiny architecture. The training process was also combined with data augmentation to provide more information for the model to learn. Then the accuracy results of each model were compared to find out the best model for the OMR solution in this research. This experiment uses the Andrea dataset and Attwenger dataset which both get the best result by using the augmentation method and ConvNeXt-Tiny as the model. The best accuracy for the Andrea dataset is 98.15% and for the Attwenger dataset is 98.43%.

show abstract

“…The experimental results show that the effect is superior to most traditional methods, even when no postprocessing is used. Pinheiro et al [ 4 ] compared CNNs to the classical deep learning networks for note recognition tasks. Rebelo et al [ 2 ] used neural networks for the first time in the note classification stage, comparing them to SVM and other models; while the results were not favorable, they laid the groundwork for future research.…”

Section: Introductionmentioning

confidence: 99%

Improved Feature Pyramid Convolutional Neural Network for Effective Recognition of Music Scores

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Music written by composers and performed by multidimensional instruments is an art form that reflects real-life emotions. Historically, people disseminated music primarily through sheet music recording and oral transmission. Among them, recording music in sheet music form was a great musical invention. It became the carrier of music communication and inheritance, as well as a record of humanity's magnificent music culture. The advent of digital technology solves the problem of difficult musical score storage and distribution. However, there are many drawbacks to using data in image format, and extracting music score information in editable form from image data is currently a challenge. An improved convolutional neural network for musical score recognition is proposed in this paper. Because the traditional convolutional neural network SEGNET misclassifies some pixels, this paper employs the feature pyramid structure. Use additional branch paths to fuse shallow image details, shallow texture features that are beneficial to small objects, and high-level features of global information, enrich the multi-scale semantic information of the model, and alleviate the problem of the lack of multiscale semantic information in the model. Poor recognition performance is caused by semantic information. By comparing the recognition effects of other models, the experimental results show that the proposed musical score recognition model has a higher recognition accuracy and a stronger generalization performance. The improved generalization performance allows the musical score recognition method to be applied to more types of musical score recognition scenarios, and such a recognition model has more practical value.

show abstract

A Deep Approach for Handwritten Musical Symbols Recognition

Cited by 8 publications

References 9 publications

Deep Multilevel Cascade Residual Recurrent Framework (MCRR) for Sheet Music Recognition

Deep Multilevel Cascade Residual Recurrent Framework (MCRR) for Sheet Music Recognition

Recognition of music symbol notation using convolutional neural network

Improved Feature Pyramid Convolutional Neural Network for Effective Recognition of Music Scores

Contact Info

Product

Resources

About