Deep Models for Ensemble Touch-Screen Improvisation

Abstract

For many, the pursuit and enjoyment of musical performance goes hand-in-hand with collaborative creativity, whether in a choir, jazz combo, orchestra, or rock band. However, few musical interfaces use the affordances of computers to create or enhance ensemble musical experiences. One possibility for such a system would be to use an artificial neural network (ANN) to model the way other musicians respond to a single performer. Some forms of music have well-understood rules for interaction; however, this is not the case for free improvisation with new touch-screen instruments where styles of interaction may be discovered in each new performance. This paper describes an ANN model of ensemble interactions trained on a corpus of such ensemble touch-screen improvisations. The results show realistic ensemble interactions and the model has been used to implement a live performance system where a performer is accompanied by the predicted and sonified touch gestures of three virtual players. CCS CONCEPTS • Computing methodologies → Neural networks; • Applied computing → Sound and music computing;

Deep Predictive Models in Interactive Music

“…The Neural Touchscreen Ensemble [51], a system developed by the authors, is an RNN-driven simulation of a touchscreen ensemble experience. A human performer plays freely improvised music on a touchscreen and an ensemble performance is continually played back on three RNN-controlled touchscreen devices in response.…”