A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions

Ji, Shulei; Luo, Jing; Yang, Xinyu

doi:10.48550/arxiv.2011.06801

Cited by 33 publications

(38 citation statements)

References 160 publications

(298 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In music generation, sequence models with symbolic music representation have exhibited considerable success [12,19]. In particular, the combination of the Transformer model [25] and a token representation of music has been revealed to generate coherent and structural music [11,14]. However, the music representations used in prior studies were note-level (MIDI-equivalent); hence, their expressions were limited compared to those of musical scores.…”

Section: (B)mentioning

confidence: 99%

Score Transformer: Generating Musical Score from Note-level Representation

Suzuki

2021

ACM Multimedia Asia

View full text Add to dashboard Cite

In this paper, we explore the tokenized representation of musical scores using the Transformer model to automatically generate musical scores. Thus far, sequence models have yielded fruitful results with note-level (MIDI-equivalent) symbolic representations of music. Although the note-level representations can comprise sufficient information to reproduce music aurally, they cannot contain adequate information to represent music visually in terms of notation. Musical scores contain various musical symbols (e.g., clef, key signature, and notes) and attributes (e.g., stem direction, beam, and tie) that enable us to visually comprehend musical content. However, automated estimation of these elements has yet to be comprehensively addressed. In this paper, we first design score token representation corresponding to the various musical elements. We then train the Transformer model to transcribe note-level representation into appropriate music notation. Evaluations of popular piano scores show that the proposed method significantly outperforms existing methods on all 12 musical aspects that were investigated. We also explore an effective notation-level token representation to work with the model and determine that our proposed representation produces the steadiest results. CCS CONCEPTS• Applied computing → Sound and music computing.

show abstract

Section: (B)mentioning

confidence: 99%

Score Transformer: Generating Musical Score from Note-level Representation

Suzuki

2021

ACM Multimedia Asia

View full text Add to dashboard Cite

show abstract

“…In recent years, deep neural networks (DNNs) have obtained significant improvement in varieties of artificial intelligence areas such as image recognition [1], neural language translation [2], automatic speech recognition [3], and automatic music generation [4]. Automatic music generation has two mainstream music generation approaches include audio music generation [5,6,7] and symbolic music generation [8,9,10].…”

Section: Introductionmentioning

confidence: 99%

“…a 4/4 musical piece. Because the musical note distribution is dominated by the holding state [4], which negatively affects the diversity of the generated notes, cosine distance is also designed as part of loss functions to prevent EMN from remembering the holding state instead of musical notes to enrich the diversity of musical notes.…”

Section: Introductionmentioning

confidence: 99%

Enhanced Memory Network: The novel network structure for Symbolic Music Generation

Li,

Liu,

Yan

et al. 2021

Preprint

View full text Add to dashboard Cite

Symbolic melodies generation is one of the essential tasks for automatic music generation. Recently, models based on neural networks have had a significant influence on generating symbolic melodies. However, the musical context structure is complicated to capture through deep neural networks. Although long short-term memory (LSTM) is attempted to solve this problem through learning order dependence in the musical sequence, it is not capable of capturing musical context with only one note as input for each time step of LSTM. In this paper, we propose a novel Enhanced Memory Network (EMN) with several recurrent units, named Enhanced Memory Unit (EMU), to explicitly modify the internal architecture of LSTM for containing music beat information and reinforces the memory of the latest musical beat through aggregating beat inside the memory gate. In addition, to increase the diversity of generated musical notes, cosine distance among adjacent time steps of hidden states is considered as part of loss functions to avoid a high similarity score that harms the diversity of generated notes. Objective and subjective evaluation results show that the proposed method achieves state-of-the-art performance. Code and music demo are available at https://github.com/qrqrqrqr/EMU

show abstract

“…Research on automatic symbolic music generation or style transfer has seen significant progress in recent years [5]- [11]. Among such efforts, the adoption of the Transformer decoderbased neural network [12] as the backbone generative model has become popular [13]- [24].…”

Section: Introductionmentioning

confidence: 99%

Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer

Shih¹,

Wu²,

Zalkow³

et al. 2021

Preprint

View full text Add to dashboard Cite

Attention-based Transformer models have been increasingly employed for automatic music generation. To condition the generation process of such a model with a user-specified sequence, a popular approach is to take that conditioning sequence as a priming sequence and ask a Transformer decoder to generate a continuation. However, this prompt-based conditioning cannot guarantee that the conditioning sequence would develop or even simply repeat itself in the generated continuation. In this paper, we propose an alternative conditioning approach, called theme-based conditioning, that explicitly trains the Transformer to treat the conditioning sequence as a thematic material that has to manifest itself multiple times in its generation result. This is achieved with two main technical contributions. First, we propose a deep learning-based approach that uses contrastive representation learning and clustering to automatically retrieve thematic materials from music pieces in the training data. Second, we propose a novel gated parallel attention module to be used in a sequence-to-sequence (seq2seq) encoder/decoder architecture to more effectively account for a given conditioning thematic material in the generation process of the Transformer decoder. We report on objective and subjective evaluations of variants of the proposed Theme Transformer and the conventional promptbased baseline, showing that our best model can generate, to some extent, polyphonic pop piano music with repetition and plausible variations of a given condition.

show abstract

A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions

Cited by 33 publications

References 160 publications

Score Transformer: Generating Musical Score from Note-level Representation

Score Transformer: Generating Musical Score from Note-level Representation

Enhanced Memory Network: The novel network structure for Symbolic Music Generation

Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer

Contact Info

Product

Resources

About