Image Synthesis via Semantic Composition

Abstract

In this paper, we present a novel approach to synthesize realistic images based on their semantic layouts. It hypothesizes that for objects with similar appearance, they share similar representation. Our method establishes dependencies between regions according to their appearance correlation, yielding both spatially variant and associated representations. Conditioning on these features, we propose a dynamic weighted network constructed by spatially conditional computation (with both convolution and normalization). More than preserving semantic distinctions, the given dynamic network strengthens semantic relevance, benefiting global structure and detail synthesis. We demonstrate that our method gives the compelling generation performance qualitatively and quantitatively with extensive experiments on benchmarks 1 .

SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing

“…In the layout-to-image translation problem, a layout image is provided as the condition for controllable image synthesis. The layout image can be a semantic segmentation mask [14,15,43,50,54,69,70,77,78], a sketch image [16,56,69], etc. Among these, some studies have attempted to represent different semantic parts with latent codes [14,77,78].…”

“…Each semantic part is modulated individually with corresponding local latent codes and an image is synthesized by composing local feature maps. Different from layout-to-image translation methods [14,70,77], our local latent codes are able to control both the structure and texture of semantic parts. Compared to attribute-conditional GANs [18,39,62], our model is not designed for any specific task and can serve as a generic prior like StyleGAN.…”