Audio Texture Synthesis with Random Neural Networks: Improving Diversity and Quality

Antognini, Joseph M.; Hoffman, Matt; Weiss, Ron

doi:10.1109/icassp.2019.8682598

Cited by 8 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig. 8 presents the results of sound texture synthesis using c-cgCNN, McDermott's model [14] and Antognini's model [29] in waveforms. Unlike other two methods which act on frequency domain, c-cgCNN only uses raw audios.…”

Section: F Results On Texture Synthesismentioning

confidence: 99%

“…As for sound texture synthesis, classic models [14] are generally based on wavelet framework and use handcrafted filters to extract temporal statistics. Recently, Antognini et al [29] extended Gatys' method to sound texture synthesis by applying a random network to the spectrograms of sound textures. In contrast, our model learns the network adaptively instead of fixing it to random weights, and our model is applied to raw waveforms directly.…”

Section: Related Workmentioning

confidence: 99%

“…We use atrous [37] convolutions to ensure large receptive fields, which enable us to learn long-range dependencies. Unlike Antognini's model [29] which applies fixed random ConvNets to the spectrograms, our model learns the ConvNet using raw waveforms.…”

Section: A Texture Synthesismentioning

confidence: 99%

“…8. Comparison of c-cgCNN, McDermott's[14] and Antognini's model[29] for sound texture synthesis. Their results are comparable.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Conditional Generative ConvNets for Exemplar-based Texture Synthesis

Wang

Xia³

2019

Preprint

View full text Add to dashboard Cite

The goal of exemplar-based texture synthesis is to generate texture images that are visually similar to a given exemplar. Recently, promising results have been reported by methods relying on convolutional neural networks (ConvNets) pretrained on large-scale image datasets. However, these methods have difficulties in synthesizing image textures with non-local structures and extending to dynamic or sound textures. In this paper, we present a conditional generative ConvNet (cgCNN) model which combines deep statistics and the probabilistic framework of generative ConvNet (gCNN) model. Given a texture exemplar, the cgCNN model defines a conditional distribution using deep statistics of a ConvNet, and synthesize new textures by sampling from the conditional distribution. In contrast to previous deep texture models, the proposed cgCNN dose not rely on pre-trained ConvNets but learns the weights of ConvNets for each input exemplar instead. As a result, the cgCNN model can synthesize high quality dynamic, sound and image textures in a unified manner. We also explore the theoretical connections between our model and other texture models. Further investigations show that the cgCNN model can be easily generalized to texture expansion and inpainting. Extensive experiments demonstrate that our model can achieve better or at least comparable results than the state-of-the-art methods.

show abstract

Section: F Results On Texture Synthesismentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A Texture Synthesismentioning

confidence: 99%

“…8. Comparison of c-cgCNN, McDermott's[14] and Antognini's model[29] for sound texture synthesis. Their results are comparable.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Conditional Generative ConvNets for Exemplar-based Texture Synthesis

Wang

Xia³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The features passed from one layer to another are called as deep features. In [42] and [43] deep features are extracted and analyzed from a convolution neural network (CNN).…”

Section: Deep Features For Texture Analysismentioning

confidence: 99%

Trends in Audio Texture Analysis, Synthesis, and Applications

Sharma¹,

Umapathy²

2022

J. Audio Eng. Soc.

View full text Add to dashboard Cite

Audio signals are classified into speech, music, and environmental sounds. From the evolution of audio features, an adequate amount of work has been seen in speech and music processing. On the other hand, the environmental sounds have not been studied that much, and the major reason behind it is the lack of coherent information present in an environmental sound compared with the speech signal or a musical sound. The definition to express audio textures is imprecise and insufficient, so audio textures tend to be defined by drawing a comparison to the known sound source (e.g., "it sounds like a motor" or "like a fan"). Audio textures could be either natural or artificial. Natural audio textures, such as heavy rain, fire, and stream flowing, are very common. The artificial audio textures include sounds such as applause, a motor running, someone walking on gravel, babble, and many more. Although these audio textures have been used in virtual reality, music, screen saver sounds, and more, a considerable amount of possible work is still untouched. The aim of this study is to summarize the literature on audio textures, textural features, and their applications. In this survey, the texture synthesis and features are explained in detail.

show abstract

Sound Model Factory: An Integrated System Architecture for Generative Audio Modelling

Wyse

Kamath

Gupta

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We introduce a new system for data-driven audio sound model design built around two different neural network architectures, a Generative Adversarial Network(GAN) and a Recurrent Neural Network (RNN), that takes advantage of the unique characteristics of each to achieve the system objectives that neither is capable of addressing alone. The objective of the system is to generate interactively controllable sound models given (a) a range of sounds the model should be able to synthesize, and (b) a specification of the parametric controls for navigating that space of sounds. The range of sounds is defined by a dataset provided by the designer, while the means of navigation is defined by a combination of data labels and the selection of a sub-manifold from the latent space learned by the GAN. Our proposed system takes advantage of the rich latent space of a GAN that consists of sounds that fill out the spaces "between" real data-like sounds. This augmented data from the GAN is then used to train an RNN for its ability to respond immediately and continuously to parameter changes and to generate audio over unlimited periods of time. Furthermore, we develop a self-organizing map technique for "smoothing" the latent space of GAN that results in perceptually smooth interpolation between audio timbres. We validate this process through user studies. The system contributes advances to the state of the art for generative sound model design that include system configuration and components for improving interpolation and the expansion of audio modeling capabilities beyond musical pitch and percussive instrument sounds into the more complex space of audio textures.

show abstract

Audio Texture Synthesis with Random Neural Networks: Improving Diversity and Quality

Cited by 8 publications

References 8 publications

Conditional Generative ConvNets for Exemplar-based Texture Synthesis

Conditional Generative ConvNets for Exemplar-based Texture Synthesis

Trends in Audio Texture Analysis, Synthesis, and Applications

Sound Model Factory: An Integrated System Architecture for Generative Audio Modelling

Contact Info

Product

Resources

About