Recently, unsupervised exemplar-based image-to-image translation, conditioned on a given exemplar without the paired data, has accomplished substantial advancements.In order to transfer the information from an exemplar to an input image, existing methods often use a normalization technique, e.g., adaptive instance normalization, that controls the channel-wise statistics of an input activation map at a particular layer, such as the mean and the variance. Meanwhile, style transfer approaches similar task to image translation by nature, demonstrated superior performance by using the higher-order statistics such as covariance among channels in representing a style. In detail, it works via whitening (given a zero-mean input feature, transforming its covariance matrix into the identity). followed by coloring (changing the covariance matrix of the whitened feature to those of the style feature). However, applying this approach in image translation is computationally intensive and error-prone due to the expensive time complexity and its non-trivial backpropagation. In response, this paper proposes an end-to-end approach tailored for image translation that efficiently approximates this transformation with our novel regularization methods. We further extend our approach to a group-wise form for memory and time efficiency as well as image quality. Extensive qualitative and quantitative experiments demonstrate that our proposed method is fast, both in training and inference, and highly effective in reflecting the style of an exemplar. Finally, our code is available at https://github.com/ WonwoongCho/GDWCT.

This paper proposes a novel approach to generate multiple color palettes that reflect the semantics of input text and then colorize a given grayscale image according to the generated color palette. In contrast to existing approaches, our model can understand rich text, whether it is a single word, a phrase, or a sentence, and generate multiple possible palettes from it. For this task, we introduce our manually curated dataset called Palette-and-Text (PAT). Our proposed model called Text2Colors consists of two conditional generative adversarial networks: the text-topalette generation networks and the palette-based colorization networks. The former captures the semantics of the text input and produce relevant color palettes. The latter colorizes a grayscale image using the generated color palette. Our evaluation results show that people preferred our generated palettes over ground truth palettes and that our model can effectively reflect the given palette when colorizing an image.

Figure 1. Fitting results of our novel texture generative model. StyleUV can generate high-fidelity images and cover the diverse nature of human faces including but not limited to the faces of a baby, a black person, an elderly person, and a young woman wearing heavy makeup.

No abstract

Figure 1. The proposed framework enhances editability in diffusion models by conditioning the generation on two latent spaces, i.e., content and style. The latent codes are effectively combined to generate novel images. Our proposed sampling technique and timestep scheduling further improve controllability. (a) Magnitude of style can be controlled to translate semantic information from the style image. (b) The learned style space supports smooth interpolations while (c) PCA on the learned latent space gives disentangled attribute specific manipulation directions. Details are provided in sections D.1 and D.2 in appendix. More results can be found in Fig. 25 in appendix.