Modifying Texture of a Photograph Object, with the Use of Neural Networks Ensemble

Vasilyev, Roman; Amelin, Vladimir N.; Rashchenko, Yu.; Rashchenko, D.

doi:10.1134/s1995080218090202

Cited by 1 publication

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The premise is that it takes the style of one image and applies it to the content of another image. Style transfer has since been explored to apply textures (Vasilyev et al, 2018) or painting vignettes (Zhang et al, 2018) to photographs, as well as more novel approaches, such as use for style transfer of animated motion (Holden et al, 2017). Style transfer has also been applied to videos (Ruder et al, 2018), working frame by frame.…”

Section: Image Synthesismentioning

confidence: 99%

Artificial Synaesthesia: An exploration of machine learning image synthesis for soundscape audio visualisation

Willcox¹

View full text Add to dashboard Cite

Within soundscape research, audio visualisations are generally targeted towards scientific audiences and not produced with artistic intentions. Machine learning is utilised in this field to increase efficiency when dealing with large amounts of data, recognizing patterns, and classifying audio. Although machine learning is effective for these purposes, it also offers image synthesis capabilities which have not been taken advantage of in current audio visualisation production. This research aims to answer the question, ‘How might machine learning image synthesis be used to visualise soundscape audio?’. Through an iterative design process, a design pipeline was developed to generate visualisations of audio using Pix2Pix (Isola, et al., 2016), a conditional adversarial neural network. Through a process of extracting audio features, converting these into simple grid images and feeding them into a trained machine learning model, a new visual interpretation of the audio can be experienced in the form of images and videos. The video design outputs communicate visual change of audio through the interrelated transformation of colour, shape, detail, and size of flower-like figures. These outputs aim to bring attention to the value of soundscapes through visually demonstrating their unique qualities. The method developed has not been previously documented according to the available literature and marks an exciting exploration of the new application of machine learning image-to-image translation as a creative tool for audio visualisation.

show abstract

Section: Image Synthesismentioning

confidence: 99%

Artificial Synaesthesia: An exploration of machine learning image synthesis for soundscape audio visualisation

Willcox¹

View full text Add to dashboard Cite

show abstract

Modifying Texture of a Photograph Object, with the Use of Neural Networks Ensemble

Cited by 1 publication

References 2 publications

Artificial Synaesthesia: An exploration of machine learning image synthesis for soundscape audio visualisation

Artificial Synaesthesia: An exploration of machine learning image synthesis for soundscape audio visualisation

Contact Info

Product

Resources

About