Content and Style Disentanglement for Artistic Style Transfer

Kotovenko, Dmytro; Sanakoyeu, Artsiom; Lang, Sabine; Ommer, Björn

doi:10.1109/iccv.2019.00452

Cited by 160 publications

(90 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feature Disentanglement. In recent years, researches [5,7,11,13,14,19,29,31,32] used generative adversarial networks (GANs), which can be applied to style transfer task in some cases, to achieve image-to-image translation. A significant thought suitable for the style transfer task is that the style and content feature should be disentangled because of the domain deviation.…”

Section: Related Workmentioning

confidence: 99%

“…representation. Kotovenko et al [13] proposed a disentanglement loss to separate style and content. Kazemi et al [11] described a style and content disentangled GAN (SC-GAN) to learn a semantic representation of content and textual patterns of style.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, some researchers [5,7,11,13,14,19,26,29,31,32] have used Generative Adversarial Networks (GANs) for high-quality image-to-image translation. GAN-based methods can generate high-quality artistic works that can be considered as real.…”

Section: Introductionmentioning

confidence: 99%

“…The style and content representation are essential for the translation model. Numerous works [5,13,29,32] focused on the disentanglement of style and content for allowing models to be aware…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Arbitrary Style Transfer via Multi-Adaptation Network

Deng

Tang

Dong

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

163

122

View full text Add to dashboard Cite

Arbitrary style transfer is a significant topic with research value and application prospect. A desired style transfer, given a content image and referenced style painting, would render the content image with the color tone and vivid stroke patterns of the style painting while synchronously maintaining the detailed content structure information. Style transfer approaches would initially learn content and style representations of the content and style references and then generate the stylized images guided by these representations. In this paper, we propose the multi-adaptation network which involves two self-adaptation (SA) modules and one co-adaptation (CA) module: the SA modules adaptively disentangle the content and style representations, i.e., content SA module uses position-wise self-attention to enhance content representation and style SA module uses channel-wise self-attention to enhance style representation; the CA module rearranges the distribution of style representation based on content representation distribution by calculating the local similarity between the disentangled content and style features in a non-local fashion. Moreover, a new disentanglement loss function enables our network to extract main style patterns and exact content structures to adapt to various input images, respectively. Various qualitative and quantitative experiments demonstrate that the proposed multi-adaptation network leads to better results than the state-of-the-art style transfer methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Arbitrary Style Transfer via Multi-Adaptation Network

Deng

Tang

Dong

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

163

122

View full text Add to dashboard Cite

show abstract

“…Style-Aware AdaIN AAMS WCT Ours w/o ASM abstract concept, it is difficult to use quantitative metrics for comprehensive measurement. Based on this fact, we introduced two types of user studies, Style Deception Score, Semantic Retention Score, with reference to [20,21,32] to perceptually evaluate the effectiveness of our algorithm. Semantic Retention Ratio.…”

Section: Ours With Asm1mentioning

confidence: 99%

Anisotropic Stroke Control for Multiple Artists Style Transfer

Chen

Yan

Liu

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Though significant progress has been made in artistic style transfer, semantic information is usually difficult to be preserved in a fine-grained locally consistent manner by most existing methods, especially when multiple artists styles are required to transfer within one single model. To circumvent this issue, we propose a Stroke Control Multi-Artist Style Transfer framework. On the one hand, we design an Anisotropic Stroke Module (ASM) which realizes the dynamic adjustment of style-stroke between the non-trivial and the trivial regions. ASM endows the network with the ability of adaptive semantic-consistency among various styles. On the other hand, we present an novel Multi-Scale Projection Discriminator to realize the texture-level conditional generation. In contrast to the single-scale conditional discriminator, our discriminator is able to capture multi-scale texture clue to effectively distinguish a wide range of artistic styles. Extensive experimental results well demonstrate the feasibility and effectiveness of our approach. Our framework can transform a photograph into different artistic style oil painting via only ONE single model. Furthermore, the results are with distinctive artistic style and retain the anisotropic semantic information. CCS CONCEPTS • Computing methodologies → Computational photography; Computer vision representations.

show abstract