Image-to-image Translation via Hierarchical Style Disentanglement

Li, Xinyang; Zhang, Shengchuan; Hu, Jie; Cao, Liujuan; Hong, Xiaopeng; Mao, Xudong; Huang, Feiyue; Wu, Yongjian; Ji, Rongrong

doi:10.1109/cvpr46437.2021.00853

Cited by 118 publications

(74 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By using a domain label to index the mapped style codes, StarGANv2 [17] learns the mixed style to translate an image of one domain to diverse images of a target domain. Using hierarchical style disentanglement, Li et al [19] proposed a translation process for controllable translations of both multi-label and multistyle. Many other semantic-level methods [55], [56], [16] have also been proposed to achieve impressive performance for recent years.…”

Section: B Facial Attribute Transfermentioning

confidence: 99%

“…Recently, many face manipulation methods have been proposed to achieve impressive performance for the manipulation of facial attributes based on the guidance information, such as geometries [14], [15], semantics [16], [17], and exemplars [18], [19]. However, in addition to manipulated regions, these face manipulation methods involve unwanted changes of colors in unedited regions, which makes these methods not feasible to facial image inpainting since users expect that the visual information of known regions should remain unchanged.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Do Inpainting Yourself: Generative Facial Inpainting Guided by Exemplars

Lu¹,

Zhao²,

Jiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Facial image inpainting is a task of filling visually realistic and semantically meaningful contents for missing or masked pixels in a face image. Although existing methods have made significant progress in achieving high visual quality, the controllable diversity of facial image inpainting remains an open problem in this field. This paper introduces EXE-GAN, a novel diverse and interactive facial inpainting framework, which can not only preserve the high-quality visual effect of the whole image but also complete the face image with exemplar-like facial attributes. The proposed facial inpainting is achieved based on generative adversarial networks by leveraging the global style of input image, the stochastic style, and the exemplar style of exemplar image. A novel attribute similarity metric is introduced to encourage networks to learn the style of facial attributes from the exemplar in a self-supervised way. To guarantee the natural transition across the boundary of inpainted regions, a novel spatial variant gradient backpropagation technique is designed to adjust the loss gradients based on the spatial location. A variety of experimental results and comparisons on public CelebA-HQ and FFHQ datasets are presented to demonstrate the superiority of the proposed method in terms of both the quality and diversity in facial inpainting.

show abstract

Section: B Facial Attribute Transfermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Do Inpainting Yourself: Generative Facial Inpainting Guided by Exemplars

Lu¹,

Zhao²,

Jiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent frameworks [93,15] unify multi-domain and multi-target i2i exploiting multiple disentangled representations. Some works [76,38], detach from literature proposing hierarchical generation. In [11], instead, they learn separately albedo and shading, regardless of the general scene.…”

Section: Disentanglement In I2imentioning

confidence: 99%

Physics-informed Guided Disentanglement in Generative Networks

Pizzati¹,

Cerri²,

Charette³

2021

Preprint

View full text Add to dashboard Cite

Image-to-image translation (i2i) networks suffer from entanglement effects in presence of physics-related phenomena in target domain (such as occlusions, fog, etc), thus lowering the translation quality and variability. In this paper, we present a comprehensive method for disentangling physics-based traits in the translation, guiding the learning process with neural or physical models. For the latter, we integrate adversarial estimation and genetic algorithms to correctly achieve disentanglement. The results show our approach dramatically increase performances in many challenging scenarios for image translation. Keywords Adversarial learning • image to image translation • feature disentanglement • adverse weather • vision and rain • GAN

show abstract

“…Among various research areas of AI, CV is a longstanding and fundamental field, which allows computers to derive meaningful information from digital images, videos, and other visual inputs. As a representative method in CV, CNN models have achieved the new SOTA performance on a wide range of tasks over the last few decades, e.g., Image Recognition [36], Object Detection [37]- [39], Image Segmentation [40]- [42], and Image Processing [43]- [46]. Image recognition involves analyzing images and identifying objects, actions, and other elements in order to draw conclusions.…”

Section: Computer Visionmentioning

confidence: 99%

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Yao¹,

Zhang²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Influenced by the great success of deep learning via cloud computing and the rapid development of edge chips, research in artificial intelligence (AI) has shifted to both of the computing paradigms, i.e., cloud computing and edge computing. In recent years, we have witnessed significant progress in developing more advanced AI models on cloud servers that surpass traditional deep learning models owing to model innovations (e.g., Transformers, Pretrained families), explosion of training data and soaring computing capabilities. However, edge computing, especially edge and cloud collaborative computing, are still in its infancy to announce their success due to the resource-constrained IoT scenarios with very limited algorithms deployed. In this survey, we conduct a systematic review for both cloud and edge AI. Specifically, we are the first to set up the collaborative learning mechanism for cloud and edge modeling with a thorough review of the architectures that enable such mechanism. We also discuss potentials and practical experiences of some on-going advanced edge AI topics including pretraining models, graph neural networks and reinforcement learning. Finally, we discuss the promising directions and challenges in this field.

show abstract

Image-to-image Translation via Hierarchical Style Disentanglement

Cited by 118 publications

References 13 publications

Do Inpainting Yourself: Generative Facial Inpainting Guided by Exemplars

Do Inpainting Yourself: Generative Facial Inpainting Guided by Exemplars

Physics-informed Guided Disentanglement in Generative Networks

Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI

Contact Info

Product

Resources

About