Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer

Abstract

Attention-based Transformer models have been increasingly employed for automatic music generation. To condition the generation process of such a model with a user-specified sequence, a popular approach is to take that conditioning sequence as a priming sequence and ask a Transformer decoder to generate a continuation. However, this prompt-based conditioning cannot guarantee that the conditioning sequence would develop or even simply repeat itself in the generated continuation. In this paper, we propose an alternative conditioning approach, called theme-based conditioning, that explicitly trains the Transformer to treat the conditioning sequence as a thematic material that has to manifest itself multiple times in its generation result. This is achieved with two main technical contributions. First, we propose a deep learning-based approach that uses contrastive representation learning and clustering to automatically retrieve thematic materials from music pieces in the training data. Second, we propose a novel gated parallel attention module to be used in a sequence-to-sequence (seq2seq) encoder/decoder architecture to more effectively account for a given conditioning thematic material in the generation process of the Transformer decoder. We report on objective and subjective evaluations of variants of the proposed Theme Transformer and the conventional promptbased baseline, showing that our best model can generate, to some extent, polyphonic pop piano music with repetition and plausible variations of a given condition.

Multi-mmlg: a novel framework of extracting multiple main melodies from MIDI files

“…Main melody is essential information of a piece of music. Main melody can be applied in various applications, including music retrieval [1,2], accompaniment generation [3,4], melody plagiarism identification [5,6], cover song recognition [7,8], and new melody generation [9][10][11]. Consequently, automatic extraction of the main melody becomes more important [12,13], and it has captivated the attention of many researchers.…”