Monophonic Music Generation With a Given Emotion Using Conditional Variational Autoencoder

Abstract

The rapid increase in the importance of human-machine interaction and the accelerating pace of life pose various challenges for the creators of digital environments. Continuous improvement of human-machine interaction requires precise modeling of the physical and emotional state of people. By implementing emotional intelligence in machines, robots are expected not only to recognize and track emotions when interacting with humans, but also to respond and behave appropriately. The machine should match its reaction to the mood of the user as precisely as possible. Music generation with a given emotion can be a good start to fulfilling such a requirement. This article presents the process of building a system generating music content of a specified emotion. As the emotion labels, four basic emotions: happy, angry, sad, relaxed, corresponding to the four quarters of Russell's model, were used. Conditional variational autoencoder using a recurrent neural network for sequence processing was used as a generative model. The obtained results in the form of the generated music examples with a specific emotion are convincing in their structure and sound. The generated examples were evaluated with two methods, in the first using metrics for comparison with the training set and in the second using expert annotation.

Generating Polyphonic Symbolic Emotional Music in the Style of Bach Using Convolutional Conditional Variational Autoencoder

“…the label happiness refers to a group of different emotions in quadrant Q1, where arousal is high and valence is positive. A similar division of emotions into four basic categories was used, among others, in [27], [28], [41]. FIGURE 2: Russell's circumplex model [16].…”

Motif Transformer: Generating Music With Motifs

“…Many scholars have also employed other networks to solve the problem of difficulty in modeling long-structured linked music. For example, Guan et al [9] and Arora et al [10] used generative adversarial networks to generate music and Grekow et al [11] for modeling musical information using variational autoencoders. However, their achievements are still unsatisfactory.…”

“…For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ The sequence-based approach will quantify the above musical information, such as notes, bars, and rhythms, into symbolic sequences for model learning. In recent years, many scholars have made good progress in music generation using neural network models such as recurrent neural networks [7], [8], generative adversarial networks [9], [10], and variational autoencoders [11]. Music usually has obvious regularity in its overall structure, and a standard piece of music typically contains many repetitive fragments that run through the entire structure of the music rather than just being reflected in the short term.…”

Variational Autoencoders for chord sequence generation conditioned on Western harmonic music complexity

“…In [27] the authors learn an effective latent space for symbolic style-aware music generation, by applying the concept of adversarial regularization to a VAE and leveraging the music metadata information as a prior for the latent space. The latent space is conditioned with respect to tonal tension in [28], while in [29] with respect to emotions, in both cases in order to generate monophonic music.…”