Music Transformer

Huang, Cheng-Zhi Anna; Vaswani, Ashish; Uszkoreit, Jakob; Shazeer, Noam; Simon, Ian; Hawthorne, Curtis; Dai, Andrew M.; Hoffman, Matthew D.; Dinculescu, Monica; Eck, Douglas

doi:10.48550/arxiv.1809.04281

Cited by 67 publications

(115 citation statements)

References 0 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…A particularly successful class of such models are those built with the attention mechanism [5]. For example, the Transformer is an attention-based architecture that has recently produced state of the art performance in natural language processing [2], computer vision [6,7], and audio signal analysis [8,9].…”

Section: Introductionmentioning

confidence: 99%

Learning Operators with Coupled Attention

Kissas¹,

Seidman²,

Guilhoto³

et al. 2022

Preprint

View full text Add to dashboard Cite

Supervised operator learning is an emerging machine learning paradigm with applications to modeling the evolution of spatio-temporal dynamical systems and approximating general black-box relationships between functional data. We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. In our architecture, the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions, enabling us to approximate nonlinear operators even when the number of output function in the training set measurements is very small. Our formulation is accompanied by rigorous approximation theoretic guarantees on the universal expressiveness of the proposed model. Empirically, we evaluate the performance of LOCA on several operator learning scenarios involving systems governed by ordinary and partial differential equations, as well as a black-box climate prediction problem. Through these scenarios we demonstrate state of the art accuracy, robustness with respect to noisy input data, and a consistently small spread of errors over testing data sets, even for out-of-distribution prediction tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning Operators with Coupled Attention

Kissas¹,

Seidman²,

Guilhoto³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The unique beat-bar hierarchical structure and long-range repetition of music suggest that the Transformerbased model might lead to better performance in music modeling. Transformer-based methods have recently been applied to music problems [34]- [37]. Huang et al [34] proposed Music Transformer to generate music and were the first to apply Transformer to music generation.…”

Section: B Transformer-based Methodsmentioning

confidence: 99%

“…Transformer-based methods have recently been applied to music problems [34]- [37]. Huang et al [34] proposed Music Transformer to generate music and were the first to apply Transformer to music generation. They showed that a Transformer-based model can generate qualified music.…”

Section: B Transformer-based Methodsmentioning

confidence: 99%

“…3) Effects of the backbone model: To validate the performance of our backbone model, Transformer-XL (TF-XL), and its attention mechanism, we first compared the transferred music of TF-XL, CNN, RNN, and LSTM and then compared different attention mechanisms mentioned in [34] based on four metrics as listed in Tables VII and VIII. In Table VII, TF-XL achieves a significantly better average overlapped area with the user's favorite music in terms of D P , D N , indicating better performance compared to other backbone models.…”

Section: F Ablation Studymentioning

confidence: 99%

See 1 more Smart Citation

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

Hu¹,

Liu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

User preference music transfer (UPMT) is a new problem in music style transfer that can be applied to many scenarios but remains understudied. Transferring an arbitrary song to fit a user's preferences increases musical diversity and improves user engagement, which can greatly benefit individuals' mental health. Most music style transfer approaches rely on datadriven methods. In general, however, constructing a large training dataset is challenging because users can rarely provide enough of their favorite songs. To address this problem, this paper proposes a novel hybrid method called User Preference Transformer (UP-Transformer) which uses prior knowledge of only one piece of a user's favorite music. Based on the distribution of music events in the provided music, we propose a new favorite-aware loss function to fine-tune the Transformer-based model. Two steps are proposed in the transfer phase to achieve UPMT based on the extracted music pattern in a user's favorite music. Additionally, to alleviate the problem of evaluating melodic similarity in music style transfer, we propose a new concept called pattern similarity (PS) to measure the similarity between two pieces of music. Statistical tests indicate that the results of PS are consistent with the similarity score in a qualitative experiment. Furthermore, experimental results on subjects show that the transferred music achieves better performance in musicality, similarity, and user preferences.

show abstract

“…Especially the Transformer architecture (Vaswani et al, 2017), popularized in the context of Natural Language processing (Brown et al, 2020) and then successfully ap-plied in several other Machine Learning tasks (Dosovitskiy et al, 2021;Lample & Charton, 2019;Biggio et al, 2021), has proven to be a powerful tool for musical sequence modelling. Initial breakthroughs by Huang et al (2018) and Payne (2019) applied language modelling techniques to symbolic music to achieve state-of-the-art music generation. These models featured limited controllability, if any, and subsequent work attempts to improve on this limitation through various avenues (Ens & Pasquier, 2020;Choi et al, 2020;Wu & Yang, 2021;Hadjeres & Crestel, 2020).…”

Section: Introductionmentioning

confidence: 99%

FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control

Dimitri¹,

Biggio²,

Kilcher³

et al. 2022

Preprint

View full text Add to dashboard Cite

Generating music with deep neural networks has been an area of active research in recent years. While the quality of generated samples has been steadily increasing, most methods are only able to exert minimal control over the generated sequence, if any. We propose the self-supervised description-to-sequence task, which allows for fine-grained controllable generation on a global level. We do so by extracting high-level features about the target sequence and learning the conditional distribution of sequences given the corresponding high-level description in a sequence-tosequence modelling setup. We train FIGARO (FIne-grained music Generation via Attentionbased, RObust control) by applying descriptionto-sequence modelling to symbolic music. By combining learned high level features with domain knowledge, which acts as a strong inductive bias, the model achieves state-of-the-art results in controllable symbolic music generation and generalizes well beyond the training distribution.

show abstract

Music Transformer

Cited by 67 publications

References 0 publications

Learning Operators with Coupled Attention

Learning Operators with Coupled Attention

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control

Contact Info

Product

Resources

About