A Content Transformation Block for Image Style Transfer

Abstract

Style transfer has recently received a lot of attention, since it allows to study fundamental challenges in image understanding and synthesis. Recent work has significantly improved the representation of color and texture and computational speed and image resolution. The explicit transformation of image content has, however, been mostly neglected: while artistic style affects formal characteristics of an image, such as color, shape or texture, it also deforms, adds or removes content details. This paper explicitly focuses on a content-and style-aware stylization of a content image. Therefore, we introduce a content transformation module between the encoder and decoder. Moreover, we utilize similar content appearing in photographs and style samples to learn how style alters content details and we generalize this to other class details. Additionally, this work presents a novel normalization layer critical for high resolution image synthesis. The robustness and speed of our model enables a video stylization in real-time and high definition. We perform extensive qualitative and quantitative evaluations to demonstrate the validity of our approach.

Arbitrary Style Transfer via Multi-Adaptation Network

“…Feature Disentanglement. In recent years, researches [5,7,11,13,14,19,29,31,32] used generative adversarial networks (GANs), which can be applied to style transfer task in some cases, to achieve image-to-image translation. A significant thought suitable for the style transfer task is that the style and content feature should be disentangled because of the domain deviation.…”

“…In recent years, some researchers [5,7,11,13,14,19,26,29,31,32] have used Generative Adversarial Networks (GANs) for high-quality image-to-image translation. GAN-based methods can generate high-quality artistic works that can be considered as real.…”

Style-Aware Normalized Loss for Improving Arbitrary Style Transfer

“…Arbitrary style transfer methods can be classified as either non-parametric [10,11,15,26,31,50,51,54] or parametric [14,19,20,25,29,37,43,44,46,49]. Non-parametric methods find similar patches between content and style images, and transfer style based on matched patches.…”

“…Neural style transfer (NST) refers to the generation of a pastiche image P from two images C and S via a neural network, where P shares the content with C but is in the style of S. While the original NST approach of Gatys [13] optimizes the transfer model for each pair of C and S, the field has rapidly evolved in recent years to develop models that support arbitrary styles out-of-the-box. NST models can, hence, be classified based on their stylization capacity into models trained for (1) a single combination of C and S [13,23,28,32,39], (2) one S [21,27,47,48], (3) multiple fixed S [2,9,24,30,42,55], and (4) infinite (arbitrary) S [4,14,15,17,19,20,25,29,31,37,43,44]. Intuitively, the category (4) of arbitrary style transfer (AST) is the most advantageous as it is agnostic to S, allowing trained models to be adopted for diverse novel styles without re-training.…”

Anisotropic Stroke Control for Multiple Artists Style Transfer

“…Style-Aware AdaIN AAMS WCT Ours w/o ASM abstract concept, it is difficult to use quantitative metrics for comprehensive measurement. Based on this fact, we introduced two types of user studies, Style Deception Score, Semantic Retention Score, with reference to [20,21,32] to perceptually evaluate the effectiveness of our algorithm. Semantic Retention Ratio.…”