MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control

Abstract

We present a novel music generation framework for music infilling, with a user friendly interface. Infilling refers to the task of generating musical sections given the surrounding multi-track music. The proposed transformer-based framework is extensible for new control tokens as the added music control tokens such as tonal tension per bar and track polyphony level in this work. We explore the effects of including several musically meaningful control tokens, and evaluate the results using objective metrics related to pitch and rhythm. Our results demonstrate that adding additional control tokens helps to generate music with stronger stylistic similarities to the original music. It also provides the user with more control to change properties like the music texture and tonal tension in each bar compared to previous research which only provided control for track density. We present the model in a Google Colab notebook to enable interactive generation.

An interactive music infilling interface for pop music composition

“…This work is based on a pop music infilling system with multiple levels of control [3]. That system can steer each track's note density/polyphony/occupation level and bar tonal tension [4].…”

“…That system can steer each track's note density/polyphony/occupation level and bar tonal tension [4]. The original interface in [3] is a colab notebook, and to make it more accessible, a Max patch is prepared to communicate with the deep learning infilling system and better fit the composer's writing process. Compared with other interfaces [5], this one provides both track and bar control.…”