Pose-Guided Person Image Synthesis in the Non-Iconic Views

Xu, Chenyu; Fu, Yanwei; Wen, Chao; Pan, Ye; Jiang, Yu‐Gang; Xue, Xiangyang

doi:10.1109/tip.2020.3023853

Cited by 17 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model proposed in [14] is abbreviated as Taming transformer (Taming) in this section. For the pose guiding, we compare the proposed iLAT with other state-of-the-art methods retrained in the PA dataset, which include PATN [54], PN-GAN [35], PoseWarp [3], MR-Net [45] and Taming transformer [14]. As the image size of PoseWarp and MR-Net is 128 × 128, we resized the outputs for the comparison.…”

Section: Methodsmentioning

confidence: 99%

“…Critically, it is very non-trivial for all these methods to generate images of locally guided image synthesis of the non-iconic foreground. Some tentative attempts have been conducted in pose-guided synthesis [40,45] with person existing in non-ironic views. On the other hand, face editing methods are mostly based on adversarial AE based inpainting [29,49,18] and GAN inversion based methods [51,38,1].…”

Section: Related Workmentioning

confidence: 99%

“…In particular, those image synthesis tasks involving persons or portrait [6,27,28] can be applied in a wide variety of scenarios, such as advertising, games, and motion capture, etc. Most real-world image synthesis tasks only involve the local generation, which means generating pixels in certain regions, while maintaining the semantic consistency, e.g., face editing [18,1,38], pose guiding [34,53,45], and image inpainting [49,29,47,51]. Unfortunately, most works can only handle the images of 'icon-view' foreground, rather than the image synthesis of 'non-iconic view' foreground [23], concerned in this paper.…”

Section: Introductionmentioning

confidence: 99%

“…Generally, several inherent problems exist in previous works for such a task. For example, despite impressive quality of images are generated, GANs/Autoencoder(AE)-based methods [49,45,18,29] are inclined to synthesize blurry local regions, as in Fig. 1(A)-row(c).…”

Section: Introductionmentioning

confidence: 99%

“…Figure 11: More results from PA. From left to right, references, targets, PATN[54], PN-GAN[35], PoseWarp[3], MR-Net[45], Taming[14], our iLAT.…”

mentioning

confidence: 99%

See 4 more Smart Citations

The Image Local Autoregressive Transformer

Cao¹,

Hong²,

Li³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, AutoRegressive (AR) models for the whole image generation empowered by transformers have achieved comparable or even better performance to Generative Adversarial Networks (GANs). Unfortunately, directly applying such AR models to edit/change local image regions, may suffer from the problems of missing global information, slow inference speed, and information leakage of local guidance. To address these limitations, we propose a novel model -image Local Autoregressive Transformer (iLAT), to better facilitate the locally guided image synthesis. Our iLAT learns the novel local discrete representations, by the newly proposed local autoregressive (LA) transformer of the attention mask and convolution mechanism. Thus iLAT can efficiently synthesize the local image regions by key guidance information. Our iLAT is evaluated on various locally guided image syntheses, such as pose-guided person image synthesis and face editing. Both the quantitative and qualitative results show the efficacy of our model.

show abstract