Image synthesis via adversarial geometric consistency pursuit

Li, Xiang; Teng, Guowei; An, Ping; Yao, Haiyan

doi:10.1016/j.image.2021.116489

Cited by 3 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…STRAT-hat STRAT-tie LSSIM↑ IoU↑ Disp↓ User↑ LSSIM↑ IoU↑ Disp↓ User↑ LSSIM↑ IoU↑ Disp↓ User↑ ST-GAN [26] 0.5655 0.5932 0.0240 7.1% 0.4362 0.6859 0.0316 9.0% 0.2780 0.1126 0.0440 8.4% ST-GAN (+s) [26] 0.6061 0.6579 0.0198 16.8% 0.5164 0.7455 0.0235 18.2% 0.2517 0.1211 0.0393 11.3% CompGAN [1] 0.5362 0.5593 0.0279 5.2% 0.4311 0.6675 0.0343 7.4% 0.2768 0.0918 0.0506 5.5% CompGAN (+s) [1] 0.5807 0.6353 0.0216 13.8% 0.5047 0.7303 0.0246 16.5% 0.2452 0.1064 0.0425 6.7% RegGAN [21] 0.5356 0.5069 0.0299 3.3% 0.4028 0.6147 0.0371 4.5% 0.2469 0.0792 0.0603 2.8% SF-GAN [45] 0.5406 0.5472 0.0267 4.9% 0.4140 0.6521 0.0365 5.7% 0.2575 0.0885 0.0544 4.4% AGCP [24] 0.5240 0.4750 0.0347 1.4% 0.3954 0.5981 0.0386 2.7% 0.2348 0.0641 0.0640 0.7% Ours 0.6886 0.7573 0.0145 47.5% 0.5470 0.7873 0.0184 36.0% 0.2883 0.3948 0.0131 60.2%…”

Section: Methods Strat-glassesmentioning

confidence: 99%

“…We compare with previous methods [21,26], which focus on spatial transformation for image composition. We also compare with [1,24,45] which deal with both appearance and geometry inconsistency between foreground and background. In other words, they not only learn the warping parameters for the foreground object, but also adjust the appearance of foreground object.…”

Section: Comparison With Baselinesmentioning

confidence: 99%

“…In this work, we focus on solving the geometry inconsistency between foreground and background in a composite image. Previous works [1,4,21,24,26,45] on geometry inconsistency learned the warping parameter (e.g., perspective transformation) of foreground for geometric correction to create a realistic composite image (see Figure 1). Due to the lack of ground-truth annotations, these methods leverage weak supervision from real images, by using adversarial learning to enforce the produced composite images to be indistinguishable from real images.…”

Section: Introductionmentioning

confidence: 99%

“…The reasons of choosing these three application scenarios are as follows. 1) Virtual try-on has broad application prospects and was studied in previous works [21,24,26]. 2) The reasonable transformation has less uncertainty compared with other composition tasks (e.g., scene/text image synthesis [12,34]), which makes the ground-truth annotation more meaningful.…”

Section: Introductionmentioning

confidence: 99%

“…Previous works [1,4,21,24,26,45] on geometry inconsistency usually concatenate the foreground and background as input, which is delivered to the network to produce the warping parameters of the foreground. However, this type of network structure ignores the correspondence between foreground and background, which may provide useful insight for this task.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Spatial Transformation for Image Composition via Correspondence Learning

Zhang¹,

Liu²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

When using cut-and-paste to acquire a composite image, the geometry inconsistency between foreground and background may severely harm its fidelity. To address the geometry inconsistency in composite images, several existing works learned to warp the foreground object for geometric correction. However, the absence of annotated dataset results in unsatisfactory performance and unreliable evaluation. In this work, we contribute a Spatial TRAnsformation for virtual Try-on (STRAT) dataset covering three typical application scenarios. Moreover, previous works simply concatenate foreground and background as input without considering their mutual correspondence. Instead, we propose a novel correspondence learning network (CorrelNet) to model the correspondence between foreground and background using cross-attention maps, based on which we can predict the target coordinate that each source coordinate of foreground should be mapped to on the background. Then, the warping parameters of foreground object can be derived from pairs of source and target coordinates. Additionally, we learn a filtering mask to eliminate noisy pairs of coordinates to estimate more accurate warping parameters. Extensive experiments on our STRAT dataset demonstrate that our proposed CorrelNet performs more favorably against previous methods.

show abstract

Section: Methods Strat-glassesmentioning

confidence: 99%

Section: Comparison With Baselinesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Spatial Transformation for Image Composition via Correspondence Learning

Zhang¹,

Liu²,

Lu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

MT-GAN: toward realistic image composition based on spatial features

Teng

et al. 2023

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

The purpose of image composition is to combine the visual elements of different natural images to produce a natural image. The performance of most existing image composition methods drops significantly when they solve multiple issues, such as image harmonization, image blending, shadow generation, object placement, and spatial transformation. To address this problem, we propose a multitask GAN for image compositing based on spatial features, aiming to simultaneously address the geometric and appearance inconsistency. We use three related learning objective functions to train the network. Moreover, a new dataset including 7756 images with RoI region annotations is contributed to help evaluate the multitask image compositing results. Extensive experiments demonstrate that our proposed method is effective on our dataset.

show abstract

Image Composition Method Based on a Spatial Position Analysis Network

Li,

Teng,

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

Realistic image composition aims to composite new images by fusing a source object into a target image. It is a challenging problem due to the complex multi-task framework, including sensible object placement, appearance consistency, shadow generation, etc. Most existing researchers attempt to address one of the issues. Especially before compositing, there is no matching assignment between the source object and target image, which often leads to unreasonable results. To address the issues above, we consider image composition as an image generation problem and propose a deep adversarial learning network via spatial position analysis. We target the analysis network segment and classify the objects in target images. A spatial alignment network matches the segmented objects with the source objects, and predicts a sensible placement position, and an adversarial network generates a realistic composite image with the shadow and reflection of the source object. Furthermore, we use the classification information of target objects to filter out unreasonable image compositing. Moreover, we introduce a new test set to evaluate the network generalization for our multi-task image composition dataset. Extensive experimental results of the SHU (Shanghai University) dataset demonstrate that our deep spatial position analysis network remarkably enhances the compositing performance in realistic, shadow, and reflection generations.

show abstract

Image synthesis via adversarial geometric consistency pursuit

Cited by 3 publications

References 11 publications

Spatial Transformation for Image Composition via Correspondence Learning

Spatial Transformation for Image Composition via Correspondence Learning

MT-GAN: toward realistic image composition based on spatial features

Image Composition Method Based on a Spatial Position Analysis Network

Contact Info

Product

Resources

About