DualAST: Dual Style-Learning Networks for Artistic Style Transfer

Chen, Haibo; Zhao, Lei; Wang, Zhizhong; Zhang, Huiming; Zuo, Zhiwen; Li, Ailin; Xing, Wei; Lu, Dongming

doi:10.1109/cvpr46437.2021.00093

Cited by 65 publications

(74 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pioneering work of Gatys et al [2016] first demonstrated the strength of Deep Convolutional Neural Networks (DC-NNs) in artistic style transfer, where the content and style can be expressed as multi-level feature statistics extracted from the pre-trained DCNNs. Since then, extensive works have been proposed to improve the performance of artistic style transfer in several aspects, such as efficiency [Johnson et al, 2016;Ulyanov et al, 2016], quality [Li and Wand, 2016a;Ulyanov et al, 2017;Zhang et al, 2019;Chen et al, 2021b;Chen et al, 2021a], generalization [Chen and Schmidt, 2016;Li et al, 2017b;Sheng et al, 2018;, diversity [Li et al, 2017a;Ulyanov et al, 2017;Chen et al, 2021c], and controllability [Babaeizadeh and Ghiasi, 2018;Yao et al, 2019].…”

Section: Methods Based On Pre-trained Networkmentioning

confidence: 99%

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Bhardwaj

Iyer

Jalan

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Air pollution monitoring and management is one of the key challenges for urban sectors, especially in developing countries. Measuring pollution levels requires significant investment in reliable and durable instrumentation and subsequent maintenance. On the other hand, there have been many attempts by researchers to develop image-based pollution measurement models which have shown significant results and established the feasibility of the idea. But, taking image-level models to a city-level system presents new challenges, which include scarcity of high-quality annotated data and a high amount of label noise. In this paper, we present a low-cost, end-to-end system for learning pollution maps using images captured through a mobile phone. We demonstrate our system for parts of New Delhi and Ghaziabad. We use transfer learning to overcome the problem of data scarcity. We investigate the effects of label noise in detail and introduce the metric of in-interval accuracy to evaluate our models in presence of noise. We use distributed averaging to learn pollution maps and mitigate the effects of noise to some extent. We also develop haze-based interpretable models which have comparable performance to mainstream models. With only 382 images from Delhi and Ghaziabad and single-scene dataset from Beijing and Shanghai, we are able to achieve a mean absolute error of 44 ug/m^3 in PM2.5 concentration on a test set of 267 images and an in-interval accuracy of 67% on predictions. Going further, we learn pollution maps with a mean absolute error as low as 35 ug/m^3 and in-interval accuracy as high as 74% significantly mitigating the image models' error. We also show that the noise in pollution labels emerging from unreliable sensing instrumentation forms a significant barrier to the realization of an ideal air pollution monitoring system. Our codebase can be found at https://github.com/ankitbha/pollution_with_images.

show abstract

Section: Methods Based On Pre-trained Networkmentioning

confidence: 99%

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Bhardwaj

Iyer

Jalan

et al. 2022

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…The seminal works of Bethge 2016, 2015) have proved the power of Deep Convolutional Neural Networks (DC-NNs) (Simonyan and Zisserman 2014) in style transfer and texture synthesis, where the Gram matrices of the features extracted from different layers of DCNNs are used to represent the style of images. Further works improved it in many aspects, including efficiency (Johnson, Alahi, and Fei-Fei 2016), quality (Jing et al 2018;Kolkin, Salavon, and Shakhnarovich 2019;Park and Lee 2019;Wang et al 2020bWang et al , 2021Chen et al 2020Chen et al , 2021bAn et al 2021), generality (Li et al 2017;Huang and Belongie 2017;Zhang, Zhu, and Zhu 2019;Jing et al 2020), and diversity (Wang et al 2020a;Chen et al 2021c). For interactive style transfer, (Gatys et al 2017) introduced user spatial control into (Gatys, Ecker, and Bethge 2016), which is further accelerated by (Lu et al 2017).…”

Section: Related Workmentioning

confidence: 99%

Texture Reformer: Towards Fast and Universal Interactive Texture Transfer

Wang

Zhao

Chen

et al. 2022

AAAI

Self Cite

View full text Add to dashboard Cite

In this paper, we present the texture reformer, a fast and universal neural-based framework for interactive texture transfer with user-specified guidance. The challenges lie in three aspects: 1) the diversity of tasks, 2) the simplicity of guidance maps, and 3) the execution efficiency. To address these challenges, our key idea is to use a novel feed-forward multi-view and multi-stage synthesis procedure consisting of I) a global view structure alignment stage, II) a local view texture refinement stage, and III) a holistic effect enhancement stage to synthesize high-quality results with coherent structures and fine texture details in a coarse-to-fine fashion. In addition, we also introduce a novel learning-free view-specific texture reformation (VSTR) operation with a new semantic map guidance strategy to achieve more accurate semantic-guided and structure-preserved texture transfer. The experimental results on a variety of application scenarios demonstrate the effectiveness and superiority of our framework. And compared with the state-of-the-art interactive texture transfer algorithms, it not only achieves higher quality results but, more remarkably, also is 2-5 orders of magnitude faster.

show abstract

“…Liu et al [2021b] present an adaptive attention normalization module (AdaAttN) to consider both shallow and deep features for attention score calculation. GAN-based methods [Kotovenko et al 2019a,b;Sanakoyeu et al 2018a;Svoboda et al 2020;Zhu et al 2017] have been successfully used in collection style transfer, which considers style images in a collection as a domain [Chen et al 2021b;Lin et al 2021;].…”

Section: Related Workmentioning

confidence: 99%

“…We introduce DE with adversarial loss to enable the network to learn the style distribution Recent style transfer models employ GAN [Goodfellow et al 2014] to align the distribution of generated images with specific artistic images [Chen et al 2021b;Lin et al 2021]. The adversarial loss can enhance the holistic style of the stylization results, while it strongly relies on the distribution of datasets.…”

Section: Domain Enhancementmentioning

confidence: 99%

Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

Zhang¹,

Tang²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

propose to learn style representation directly from image features instead of their second-order statistics, by analyzing the similarities and differences between multiple styles and considering the style distribution. Specifically, we present Contrastive Arbitrary Style Transfer (CAST), which is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer. We conduct qualitative and quantitative evaluations comprehensively to demonstrate that our approach achieves significantly better results compared to those obtained via state-of-the-art methods. Code and models are available at https://github.com/zyxElsa/CAST_pytorch. CCS CONCEPTS• Computing methodologies → Image processing.

show abstract

DualAST: Dual Style-Learning Networks for Artistic Style Transfer

Cited by 65 publications

References 24 publications

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Texture Reformer: Towards Fast and Universal Interactive Texture Transfer

Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

Contact Info

Product

Resources

About