Improving orchestral conducting systems in public spaces

Mapping by Observation: Building a User-Tailored Conducting System From Spontaneous Movements

“…We chose this excerpt for two reasons: first, it is a very popular melody that all participants in the study knew in advance (as they later confirmed); second, the selected melody mostly contains quarter notes. This makes the beat and rhythm of the melody equivalent and avoids possible confusions with participants tending to conduct to the onsets of the predominant melody instead of the beat (this effect was observed by Lee et al, 2005). We used symbolic score material for the synthesis since it simplifies the control of the performance in terms of event triggering and loudness manipulation, as compared to the case of using a real pre-recorded performance to manipulate.…”

“…By "spontaneous" we refer to conducting movements that the user performs on top of a musical excerpt without having received any specific instructions. In this sense, the system needs the user to "conduct" on top of a musical piece for which there is available information on the loudness and location of beats, just as in the case of previous observation studies (Lee et al, 2005;Sarasúa and Guaus, 2014a,b) (in this case, since the music is rendered from a symbolic musical score, the location of the beats and the loudness are known). More concretely, this system takes into consideration how the user tends to anticipate or fall behind the beat, and which body movement descriptors are best correlated with loudness.…”

“…However, Lee et al (2005) identified some usability breakdowns when qualitatively analyzing how people performed with their systems in public spaces (Borchers et al, 2002;Lee et al, 2004), and decided to analyze with more detail the temporal relationship between users' conducting gestures and the beat on a musical piece. In order to do that, they asked professional conductors and non-conductors to "conduct" a fixed musical clip from the Radetzky March using up-down movements making them aware that their movements were not affecting the resulting sound.…”

“…In the case of Lee et al (2005), the comparison between different expertise profiles was done using a fixed gesture (updown hand movements). We believe, however, that in the case of public installations, it is potentially more engaging to allow users to freely perform with spontaneous movements.…”

“…Similarly to the case of Lee et al (2005), we hypothesize that an interactive conducting system that adapts its parameters to the observations from spontaneous conducting movements can see its usability improved and also become more intuitive. However, following the conclusions from the aforementioned study, these adaptations must be user-specific (and not just based on different expertise profiles).…”