Enhanced Memory Network: The novel network structure for Symbolic Music Generation

Abstract

Symbolic melodies generation is one of the essential tasks for automatic music generation. Recently, models based on neural networks have had a significant influence on generating symbolic melodies. However, the musical context structure is complicated to capture through deep neural networks. Although long short-term memory (LSTM) is attempted to solve this problem through learning order dependence in the musical sequence, it is not capable of capturing musical context with only one note as input for each time step of LSTM. In this paper, we propose a novel Enhanced Memory Network (EMN) with several recurrent units, named Enhanced Memory Unit (EMU), to explicitly modify the internal architecture of LSTM for containing music beat information and reinforces the memory of the latest musical beat through aggregating beat inside the memory gate. In addition, to increase the diversity of generated musical notes, cosine distance among adjacent time steps of hidden states is considered as part of loss functions to avoid a high similarity score that harms the diversity of generated notes. Objective and subjective evaluation results show that the proposed method achieves state-of-the-art performance. Code and music demo are available at https://github.com/qrqrqrqr/EMU

Toward Interactive Music Generation: A Position Paper

“…The authors in [87] proposed a novel technique to improve the performance of LSTM RNN models to learn longtemporal dependencies. The proposed model is named Enhanced Memory Network (EMN), which consists of several recurrent units known as Enhanced Memory Units (EMU).…”

“…Comparably, models such as [127] provide interactive and controllable generation through the captured latent space. 2019 Single-track Polyphonic Normalizing Flows/LSTM-RNN ✓ ✓ 11 [20] 2019 Melody LSTM-RNN ✓ ✓ 12 Two-stageRNN [8] 2019 Melody LSTM-RNN ✓ ✓ ✓ 13 [29] 2021 Melody LSTM-RNN ✓ ✓ ✓ ✓ ✓ ✓ 14 [87] 2021 Melody LSTM-RNN ✓ ✓ 15 [47] 2021 Single-track Polyphonic LSTM-RNN ✓ ✓ ✓ ✓ 16 MeloForm [96] 2022 Indeed, interactivity allows artists to perform local modifications and regenerate specific musical parts incrementally. This functionality is essential for the music generation systems to be practical and assist artists in composing music.…”

Melody Harmonization with Controllable Harmonic Rhythm

“…AutoHarmonizer trained/validated on a lead sheet version of Nottingham Music Database 2 (NMD), a collection of 1,034 British folk songs. NMD has been appearing more and more in machine learning researches [6,20,36] for music over the past few years. However, these tunes are not originally checked by hand, so there are some mistakes in this dataset.…”