No abstract

This paper summarizes the results of the design and implementation of an audio-visual installation, which focuses on the interaction of two casual players with complementary designed musical sphere interfaces. While one user interacts with a small dome touchscreen and controls the harmonic sound space, the other user stands inside a larger dome creating melodies by continuous gestures in mid-air. Similar to a theremin notes are played without touch using an invisible two-dimensional melodic table of notes with a dedicated layout that allows creating harmonic expressions by continuous gestures. The generation of musical expressions is visualized on both instruments with several visualization techniques. For prototyping purposes and to illustrate the idea of the installation without the need to set up the complex installation we also provide an immersion user experience by simulating the system using an Oculus Rift head-mounted display. The goal of this project is to provide and explore new methods of musical interaction and to exploit the complementary character of the installation to create a harmonic sound environment based on cooperative play.

In this work, a rapid prototyping approach for natural real-time interactions in virtual studio environments was used, in order to facilitate the creation of an interactive virtual robotic arm, that could be controlled by hand movements in a natural manner. The approach focuses on tracking a person and manipulating control parameters as a way of steering interactions. It has two main aspects. The first one is the authoring of spatial relationships between the person and the virtual environment. The second aspect is the use of gesture following to synchronize various animations or event sequences inside the virtual environment to the corresponding person's movements. Both aspects allow for a programming-by-demonstration approach and, combined, enable developers to rapidly create interactions by providing examples directly inside the blue or green box of the virtual studio.

No abstract

No abstract