DrawingInStyles: Portrait Image Generation and Editing With Spatially Conditioned StyleGAN

Su, Wanchao; Ye, Hui; Chen, Shuyu; Gao, Lin; Fu, Hongbo

doi:10.1109/tvcg.2022.3178734

Cited by 9 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GAN inversion involves manipulating real images by identifying controllable directions in the latent space, obviating the necessity for dedicated paired supervision data to train an independent network. Su et al [17] deleted the low-level generator module, mapped the sketch directly to the middle layer of the generator, and realized the editing of the building through the sketch. In addition, Alaluf et al [18] and Dinh et al [19] achieved more accurate reconstruction by training a smaller network to generate weights for StyleGAN.…”

Section: Architectural Image Editingmentioning

confidence: 99%

RSCAN: Residual Spatial Cross-Attention Network for High-Fidelity Architectural Image Editing by Fusing Multi-Latent Spaces

Zhu,

Zhao,

Lin

et al. 2024

Electronics

View full text Add to dashboard Cite

Image editing technology has brought about revolutionary changes in the field of architectural design, garnering significant attention in both the computer and architectural industries. However, architectural image editing is a challenging task due to the complex hierarchical structure of architectural images, which complicates the learning process for the high-dimensional features of architectural images. Some methods invert the images into the latent space of a pre-trained generative adversarial network (GAN) model, completing the editing process by manipulating this latent space. However, the task of striking a balance between reconstruction fidelity and editing efficacy through latent space mapping presents a formidable challenge. To address this issue, we propose a Residual Spatial Cross-Attention Network (RSCAN) for architectural image editing, which is an encoder model integrating multiple latent spaces. Specifically, we introduce the spatial feature extractor, which maps the image to the high-dimensional space F of the synthesis network, to enhance the spatial information retention and preserve the structural consistency of the architectural image. In addition, we propose the residual cross-attention to learn the mapping relationship between the low-dimensional space W and F space, generating modified features corresponding to the latent code and leveraging the benefits of multiple latent spaces to facilitate editing. Extensive experiments are performed on the LSUN Church dataset, and the experimental results indicate that our proposed RSCAN achieves significant improvements over the relevant methods in quantitative analysis metrics including the reconstruction quality, SSIM, FID, L2, LPIPS, PSNR, and editing effect ΔS, with enhancements of 29.49%, 17.29%, 8.81%, 11.43%, 11.26%, and 47.8%, respectively, thereby enhancing the practicality of architectural image editing.

show abstract

Section: Architectural Image Editingmentioning

confidence: 99%