USIS: Unsupervised Semantic Image Synthesis

Eskandar, George; Abdelsamad, Mohamed; Armanious, Karim; Yang, Bin

doi:10.1016/j.cag.2022.12.010

Cited by 11 publications

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unsupervised baselines aims to implement a translation of semantic maps to realistic images using unpaired training data. Unsupervised baseline models include CycleGAN 43 , DistanceGAN 44 , MUNIT 45 , DRIT 46 , GCGAN 47 , CUT 48 , USIS 49 and so on. In contrast, supervised baselines can produce higher quality images by utilising input data with labels.…”

Section: Experiments Experimental Settingsmentioning

confidence: 99%

UNet-like network fused Swin Transformer and CNN for Semantic Image Synthesis

Ke,

Luo,

Cai

2024

Preprint

View full text Add to dashboard Cite

Semantic image synthesis approaches has been dominated by the modelling of Convolutional Neural Networks (CNN). Due to the limitations of local perception, their performance improvement seems to have plateaued in recent years. To tackle this issue, we propose the TransUNet model, which is a UNet-like network fused Swin Transformer and CNN for semantic image synthesis. Photorealistic image synthesis conditional on the given semantic layout depends on the high-level semantics and the low-level positions. To improve the synthesis performance, we design a novel conditional residual fusion module for the model decoder to efficiently fuse the hierarchical feature maps extracted at different scales. Moreover, this module combines the opposition-based learning mechanism and the weight assignment mechanism for enhancing and attending the semantic information. Compared to pure CNN-based models, our TransUNet combines the local and global perceptions to better extract high- and low-level features and better fuse multi-scale features. We have conducted an extensive amount of comparison experiments, both in quantitative and qualitative terms, to validate the effectiveness of our proposed TransUNet model for semantic image synthesis. The outcomes illustrate that TransUNet distinctively outperforms the state-of-the-art model on three benchmark datasets (Citysacpes, ADE20K, and COCO-Stuff) including numerous real-scene images.

show abstract