Controlling Perceptual Factors in Neural Style Transfer

Abstract

Neural Style Transfer has shown very exciting results enabling new forms of image manipulation. Here we extend the existing method to introduce control over spatial location, colour information and across spatial scale 12 . We demonstrate how this enhances the method by allowing high-resolution controlled stylisation and helps to alleviate common failure cases such as applying ground textures to sky regions. Furthermore, by decomposing style into these perceptual factors we enable the combination of style information from multiple sources to generate new, perceptually appealing styles from existing ones. We also describe how these methods can be used to more efficiently produce large size, high-quality stylisation. Finally we show how the introduced control measures can be applied in recent methods for Fast Neural Style Transfer.

Transforming photos to comics using convolutional neural networks

“…Our method (as well as [1]) does not take semantic information into account, so it may transfer semantically different regions from the style images to target images. This could be improved by using semantic maps [18,19]. Presentation ability of our model.…”

Art in the Age of Machine Intelligence

“…For now, we will briefly introduce the two specific technologies used in our first gallery event, Deep Dream (in partnership with Gray Area Foundation for the Arts in San Francisco, http://grayarea.org/event/deepdreamthe-art-of-neural-networks) 7 . These are "Inceptionism" or "Deep Dreaming", first developed by Alex Mordvintsev at Google's Zurich office (Mordvintsev et al 2015), and "style transfer", first developed by Leon Gatys and collaborators in the Bethge Lab at the Centre for Integrative Neuroscience in Tübingen (Gatys et al 2016). It is fitting and likely a sign of things to come that one of these developments came from a computer scientist working on a neurally inspired algorithm for image classification, while the other came from a grad student in neuroscience working on computational models of the brain.…”

Deep Painterly Harmonization

“…However, this technique is sensitive to mismatches in the image content and several approaches have been proposed to address this issue. Gatys et al [GEB*17] add the possibility for users to guide the transfer with annotations. In the context of photographic transfer, Luan et al [LPSB17] limit mismatches using scene analysis.…”