Mobile System for Optical Music Recognition and Music Sound Generation

Abstract

Part 9: Music Information Processing WorkshopInternational audienceThe paper presents a mobile system for generating a melody based on a photo of a musical score. The client-server architecture was applied. The client role is designated to a mobile application responsible for taking a photo of a score, sending it to the server for further processing and playing mp3 file received from the server. The server role is to recognize notes from the image, generate mp3 file and send it to the client application. The key element of the system is the program realizing the algorithm of notes recognition. It is based on the decision trees and characteristics of the individual symbols extracted from the image. The system is implemented in the Windows Phone 8 framework and uses a cloud operating system Microsoft Azure. It enables easy archivization of photos, recognized notes in the Music XML format and generated mp3 files. An easy transition to other mobile operating systems is possible as well as processing multiple music collections scans

Understanding Optical Music Recognition

“…Music can be conceptualized as a structure of notes in time. This is not necessarily the only way to conceptualize music, 2 but it is the only one that has a consistent, broadly accepted visual language used to transmit it in writing, so it is the conceptualization we consider for the purposes of OMR. A note is a musical object that is defined by four parameters: pitch, duration, loudness, and timbre.…”

“…6 The music notation document serves as a medium, designed to encode and transmit a musical idea from the composer to the performer, enabling the recovery and interpretation of that envisioned music by reading through it. The performer would: 2) Use this information to parse and decode the notes and their accompanying instructions (e.g., indications of which technique to use), and 3) Apply musical intuition, prior knowledge, and taste to interpret the music and fill in the remaining parameters which music notation did not capture. Note that step (3) is clearly outside of OMR since it needs to deal with information that is not written into the music document-and where human performers start to disagree, although they are reading the very same piece of music [180].…”

“…The underlying document quality is a continuum on a scale from perfect or nearly flawless (e.g., if the document was born-digital and printed) to heavily degraded or defaced documents (e.g., ancient manuscripts that deteriorated over time and exhibit faded ink, ink blots, stains, or bleedthrough) [44]. The image acquisition mode is also a continuum that can reach from born-digital images, over scans of varying quality to low-quality, distorted photos that originate from camera-based scenarios with handheld cameras, such as smartphones [2], [300].…”

“…• technology which transforms sheet music or printed scores into a machine readable format [1] • automatic recognition and classification of symbolic music notation [2] • system that aims to minimise human involvement in music input. The musical score is scanned to a bitmap image, and the computer attempts to parse the bitmap [5] • form of structured document analysis where symbols overlaid on the conventional five-line stave are isolated and identified so that the music can be played through a MIDI system, or edited in a music publishing system [8] • identifying musical symbols on a scanned sheet of music, and interpreting them so that the music can either be played by the computer, or put into a music editor [9] • system to convert optically scanned pages of music into a versatile machine-readable format [7] • system that aims at converting optically scanned pages of music into a versatile machine-readable format [13] • system that aims at converting the vast repositories of sheet music in the world into an on-line digital format [15] • computer system that can 'read' printed music [10] • system that can be used to convert music scanned from paper into a format suitable for playing or editing on a computer [11] • technique that makes it possible to automatically build indexes on the actual content of sheet music [14] • process to automatically extract symbolic note information from scanned pages [16] • system to convert sheet music images to symbolic music representations [17] • the recognition of music scores [19] • field devoted to transcribe sheet music into some machine-readable format [18] • the process to convert a music score image into a machine-readable format [20] • task of transcribing a music score into a machine readable format [21] • task of recognizing and interpreting printed music and its transformation into MIDI [24] • research directed towards the recognition of printed scores as well as handwritten music notation [23] (Actually referred to as Optical Music Reading) • systems for music score recognition [26] • software that recognises music notation and produces a symbolic representation of music [27] • key problem for coding western music sheets in the digital world [29] • system that aims at saving time in converting hardcopy of the music score into an electronic version [31] • task devoted to convert an image of...…”