Semi-Supervised Image-to-Image Translation

Oza, Manan; Vaghela, Himanshu; Bagul, Sudhir

doi:10.1109/icaiit.2019.8834613

Cited by 9 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead of generation from a random noise, conditional image synthesis refers to the task of generating photo-realistic images conditioned on the input such as texts [17], [18], [19], [20] and images [1], [4], [21], [22], [23], [24]. Our work focuses on a special form of conditional image synthesis that aims at generating photo-realistic images conditioned on input segmentation masks, which is called semantic image synthesis.…”

Section: Conditional Image Synthesismentioning

confidence: 99%

Efficient Semantic Image Synthesis via Class-Adaptive Normalization

Tan¹,

Chen²,

Chu³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Spatially-adaptive normalization (SPADE) is remarkably successful recently in conditional semantic image synthesis [1], which modulates the normalized activation with spatially-varying transformations learned from semantic layouts, to prevent the semantic information from being washed away. Despite its impressive performance, a more thorough understanding of the advantages inside the box is still highly demanded to help reduce the significant computation and parameter overhead introduced by this novel structure. In this paper, from a return-on-investment point of view, we conduct an in-depth analysis of the effectiveness of this spatially-adaptive normalization and observe that its modulation parameters benefit more from semantic-awareness rather than spatial-adaptiveness, especially for high-resolution input masks. Inspired by this observation, we propose class-adaptive normalization (CLADE), a lightweight but equally-effective variant that is only adaptive to semantic class. In order to further improve spatial-adaptiveness, we introduce intra-class positional map encoding calculated from semantic layouts to modulate the normalization parameters of CLADE and propose a truly spatially-adaptive variant of CLADE, namely CLADE-ICPE. Through extensive experiments on multiple challenging datasets, we demonstrate that the proposed CLADE can be generalized to different SPADE-based methods while achieving comparable generation quality compared to SPADE, but it is much more efficient with fewer extra parameters and lower computational cost. The code is available at https://github.com/tzt101/CLADE.git

show abstract

Section: Conditional Image Synthesismentioning

confidence: 99%

Efficient Semantic Image Synthesis via Class-Adaptive Normalization

Tan¹,

Chen²,

Chu³

et al. 2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Conditional image synthesis refers to the task of generating photo-realistic images conditioned on different types of input, such as texts [10,26,32,34] and images [12,14,21,36,22,23]. In this paper, we focus on a special form of conditional image synthesis that aims at generating photo-realistic images conditioned on input segmentation masks, called semantic image synthesis.…”

Section: Conditional Image Synthesismentioning

confidence: 99%

Rethinking Spatially-Adaptive Normalization

Tan,

Chen,

Chu

et al. 2020

Preprint

View full text Add to dashboard Cite

“… Semi-Supervised Image-to-Image Translation -When there is limited paired data between the input and target datasets, it is possible to train an Image-to-Image translation GAN in a semi -supervised way. It uses the images that are labeled to uncover the content and style domain but uses the unpaired images to generalize the connection between the real images and translated images [5], [7].  Few-shot Image-to-Image Translation -Humans have the ability to train their brain on just one or a couple of images rather than requiring an entire dataset of images to uncover the content of the image.…”

Section: Introductionmentioning

confidence: 99%

Reviewing Fid and Sid Metrics on Generative Adversarial Networks

Deijn,

Batra,

Koch

et al. 2024

AI, Machine Learning and Applications

View full text Add to dashboard Cite

The growth of generative adversarial network (GAN) models has increased the ability of image processing and provides numerous industries with the technology to produce realistic image transformations. However, with the field being recently established there are new evaluation metrics that can further this research. Previous research has shown the Fréchet Inception Distance (FID) to be an effective metric when testing these image-to-image GANs in real-world applications. Signed Inception Distance (SID), a founded metric in 2023, expands on FID by allowing unsigned distances. This paper usespublic datasetsthat consist of façades, cityscapes, and maps within Pix2Pix and CycleGAN models. After trainin , these models are evaluated on both inception distance metrics which measure the generating performance of the trained models. Our findings indicate that usage of the metric SID incorporates an efficient and effective metric to complement, or even exceed the ability shown using the FID for the image-to-image GANs.

show abstract

Semi-Supervised Image-to-Image Translation

Cited by 9 publications

References 16 publications

Efficient Semantic Image Synthesis via Class-Adaptive Normalization

Efficient Semantic Image Synthesis via Class-Adaptive Normalization

Rethinking Spatially-Adaptive Normalization

Reviewing Fid and Sid Metrics on Generative Adversarial Networks

Contact Info

Product

Resources

About