Automated Page Turner for Musicians

Abstract

An increasing number of musicians are opting to use tablet devices instead of traditional print media for their music sheets since the digital medium offers the benefit of storing a lot of music in a compact space. The limited screen size of the tablet devices makes the music difficult to read and musicians often opt to display part of the music page at a time. With fewer music lines on display, the musician will then have to resort to scrolling through the music to read the entire score. This scrolling is annoying since the musicians will need to remove their hands from the instrument to interact with the tablet, causing a break in the music if this is not done quickly enough, or if the tablet is not sufficiently responsive. In this paper, we describe an alternative page turning system which automates the page turning event of the musician. By actively monitoring the musician's on-screen point of regard, the system retains the musician in the loop and thus, the page turns are attuned to the musician's position on the score. By analysing the way the musician's gaze changes between attention to the score and the instrument as well as the way musicians fixate on different parts of the score, we note that musicians often look away from the score and toward their hands, or elsewhere, when playing the instrument. As a result, the eye regions fall outside the field-of-view of the eye-gaze tracker, giving rise to erratic page-turns. To counteract this problem, we create a gaze prediction model that uses Kalman filtering to predict where the musician would be looking on the score. We evaluate our hands-free page turning system using 15 different piano songs containing different levels of difficulty, various repeats, and which also required playing in different registers on the piano, thus, evaluating the applicability of the page-turner under different conditions. Performance of the page-turner was quantified through the number of correct page turns, the number of delayed page turns, and the number of mistaken page turns. Of the 289 page turns involved in the experiment, 98.3% were successfully executed, 1.7% were delayed, while no mistaken page turns were observed.

Automatic Page-Turner for Pianists With Wearable Motion Detector

“…The high cost of mechanical parts and low performance were addressed as significant drawbacks of this generation of page-turners [1]. However, with the advancement of technology and computers in recent decades, researchers have proposed digital solutions such as audio and video processing techniques [3,4,5] to solve the page-turning problems.…”

“…Before the invention of automatic page-turners, musicians utilized different solutions to overcome the page-turning problem. For example, professional music editors would set long notes or pauses (rests) at the end of each page [5], so the musician had a short time to turn pages. Another approach was hiring a skillful person (called a page-turner) to sit close to the musician to follow notes during the performance and turn the page at the right time.…”

“…Kora [16] used Google Glass as a wearable device to identify the head motions and winks as a gesture for triggering the page-turning action. Tabone [5] worked on an automatic page-turning system based on the gaze tracking design. The proposed system used a dark pupil eye-tracking module with an infrared illumination tool (for emitting light in the infrared spectrum).…”

“…Python [1,2,4,6,7,8,9,10,14,18,19,20] predicted = [0, 2,4,7,7,7,9,11,12,17,18] distance, path = fastdtw(observed, predicted, dist=euclidean) >>> distance= 10.00 >>> path= [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5,5), (6,6), (7,7), (8,8), (9,9), (10, 10), (11,10)] kmeans1d import kmeans1d data = [1,2,3,3,4,5,6,6,7,8,9,10,14,15,15,15,…”

“…All steps of CC function for two signals as: f =[8,9,11,12,8,9,11,13] and g =[5,6,2,3,4,6,7,4] with the range of lags between [−1, 2] is shown in Table 4.3, and visualized in fig. 4.5. )…”