A transformer generative adversarial network for multi‐track music generation

The dendritic neural model (DNM) mimics the non-linearity of synapses in the human brain to simulate the information processing mechanisms and procedures of neurons. This enhances the understanding of biological nervous systems and the applicability of the model in various fields. However, the existing DNM suffers from high complexity and limited generalisation capability. To address these issues, a DNM pruning method with dendrite layer significance constraints is proposed. This method not only evaluates the significance of dendrite layers but also allocates the significance of a few dendrite layers in the trained model to a few dendrite layers, allowing the removal of low-significance dendrite layers. The simulation experiments on six UCI datasets demonstrate that our method surpasses existing pruning methods in terms of network size and generalisation performance.

show abstract

“…Its effectiveness in classification, approximation, and prediction has been demonstrated in various studies, including refs. [17, 19–40].…”

Section: Introductionmentioning

confidence: 99%

Pruning method for dendritic neuron model based on dendrite layer significance constraints

Luo

Wen

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

show abstract

“…Deep learning (DL) represents a class of cutting-edge ML algorithms inspired by neural networks. The DL models have been able to achieve near-human accuracy levels in various types of classification and prediction tasks including images, text, speech, and video data [1][2][3]. Thus the demand to capitalise on the potential shown by the DL solutions continues to grow.…”

Section: Introductionmentioning

confidence: 99%

Deep learning's fitness for purpose: A transformation problem frame's perspective

Gudaparthi

Niu

Yang

et al. 2023

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Combined sewer overflows represent significant risks to human health as untreated water is discharged to the environment. Municipalities, such as the Metropolitan Sewer District of Greater Cincinnati (MSDGC), recently began collecting large amounts of water-related data and considering the adoption of deep learning (DL) solutions like recurrent neural network (RNN) for predicting overflow events. Clearly, assessing the DL's fitness for the purpose requires a systematic understanding of the problem context. In this study, we propose a requirements engineering framework that uses the problem frames to identify and structure the stakeholder concerns, analyses the physical situations in which the highquality data assumptions may not hold, and derives the software testing criteria in the form of metamorphic relations that incorporate both input transformations and output comparisons. Applying our framework to MSDGC's overflow prediction problem enables a principled way to evaluate different RNN solutions in meeting the requirements. K E Y W O R D S deep learning, deep neural networks, software engineeringThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…Therefore, we propose a framework to efficiently and intelligently implement music generation and immersive sound field twinning for concerts in the metaverse, namely MetaMGC (Music Generation Framework for Concerts in Metaverse). It consists of three main parts: (1) a music generation part that enables improvised accompaniment of a virtual orchestra for metaverse concerts; (2) a digital audio twin part that enables virtual sound field reconstruction for metaverse concerts [6]; and (3) an audio rendering part that realizes the virtual soundstage production of the metaverse concert.…”

Section: Introductionmentioning

confidence: 99%

MetaMGC: a music generation framework for concerts in metaverse

Jin

Wang

et al. 2022

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

In recent years, there has been a national craze for metaverse concerts. However, existing meta-universe concert efforts often focus on immersive visual experiences and lack consideration of the musical and aural experience. But for concerts, it is the beautiful music and the immersive listening experience that deserve the most attention. Therefore, enhancing intelligent and immersive musical experiences is essential for the further development of the metaverse. With this in mind, we propose a metaverse concert generation framework — from intelligent music generation to stereo conversion and sound field design for virtual concert stages. First, combining the ideas of reinforcement learning and value functions, the Transformer-XL music generation network is improved and used in training all the music in the POP909 dataset. Experiments show that both improved algorithms have advantages over the original method in terms of objective evaluation and subjective evaluation metrics. In addition, this paper validates a neural rendering method that can be used to generate spatial audio based on a binaural-integrated neural network with a fully convolutional technique. And the purely data-driven end-to-end model performs to be more reliable compared with traditional spatial audio generation methods such as HRTF. Finally, we propose a metadata-based audio rendering algorithm to simulate real-world acoustic environments.

show abstract

A transformer generative adversarial network for multi‐track music generation

Cited by 30 publications

References 25 publications

Pruning method for dendritic neuron model based on dendrite layer significance constraints

Pruning method for dendritic neuron model based on dendrite layer significance constraints

Deep learning's fitness for purpose: A transformation problem frame's perspective

MetaMGC: a music generation framework for concerts in metaverse

Contact Info

Product

Resources

About