Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture

Yu, Ning; Barnes, Connelly; Shechtman, Eli; Amirghodsi, Sohrab; Lukáč, Michal

doi:10.1109/cvpr.2019.01244

Cited by 41 publications

(35 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This type of textures have relatively simple structures, as they can be feasibly modeled by Gaussian texture models [5]. In this experiment, we compare our algorithm with the GaussTexton [5], Image Melding [4] and TexMixer [32]. As GaussTexton is specifically designed for Gaussian texture mixing, and the detailed shape of the grass is completely missed.…”

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

“…This type of mixing is of particular interesting in real applications, because the mixed textures create more inner variance in a category. As the considered textures can not be modeled by Gaussian models, we only compare our method with Image Melding [4] and TexMixer [32]. Observe that our mixing algorithm can mix the edges and the shapes of pebbles simultaneously, and create smooth transitions from one exemplar texture to the other without ''ghosting''.…”

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

“…This is the most difficult scenario in texture mixing, as the exemplars contain regular textural elements and their visual properties, such as color and patterns, are significantly different. We compare our method with Image Melding, TexMixer [32] and DFN [22]. Notice all baseline methods fail to generate intermediate textures in this experiments.…”

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

“…Although these algorithms can generate homogeneous mixed textures, they have difficulties in mixing structured textures. Recently, Yu et al [32] proposed a deep model called MixNet for mixing nature textures. Although this model can handle nature textures relatively well, it can not be used for unseen images and generally takes several days to train.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Texture Mixing by Interpolating Deep Statistics via Gaussian Models

Xue

Wang

2020

IEEE Access

View full text Add to dashboard Cite

Recently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are used to describe textures. Although these models have achieved promising results, the structure of their parametric space is still unclear. Consequently, it is difficult to use them to mix textures. This paper addresses the texture mixing problem by using a Gaussian scheme to interpolate deep statistics computed from deep neural networks. More precisely, we first reveal that the statistics used in existing deep models can be unified using a stationary Gaussian scheme. We then present a novel algorithm to mix these statistics by interpolating between Gaussian models using optimal transport. We further apply our scheme to Neural Style Transfer, where we can create mixed styles. The experiments demonstrate that our method outperforms a number of baselines. Because all the computations are implemented in closed forms, our mixing algorithm adds only negligible time to the original texture synthesis procedure. INDEX TERMS Texture modeling, texture mixing, Gaussian models, deep neural networks.

show abstract

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

Section: B Comparisons With State-of-the-art Texture Mixing Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Texture Mixing by Interpolating Deep Statistics via Gaussian Models

Xue

Wang

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Yu et al [32] proposed a new method that can control texture interpolation called Texture Mixer. The structure of Texture Mixer is shown in Fig.…”

Section: ) Texture Mixermentioning

confidence: 99%

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

et al. 2020

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) have achieved impressive results in various image synthesis tasks, and are becoming a hot topic in computer vision research because of the impressive performance they achieved in various applications. In this paper, we introduce the recent research on GANs in the field of image processing, including image synthesis, image generation, image semantic editing, image-to-image translation, image super-resolution, image inpainting, and cartoon generation. We analyze and summarize the methods used in these applications which have improved the generated results. Then, we discuss the challenges faced by GANs and introduce some methods to deal with these problems. We also preview some likely future research directions in the field of GANs, such as video generation, facial animation synthesis and 3D face reconstruction. The purpose of this review is to provide insights into the research on GANs and to present the various applications based on GANs in different scenarios. INDEX TERMS Generative adversarial networks, image synthesis, image-to-image translation, image editing, cartoon generation.

show abstract

Wasserstein Generative Models for Patch-Based Texture Synthesis

Houdard

Leclaire

Papadakis

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper, we propose a framework to train a generative model for texture image synthesis from a single example. To do so, we exploit the local representation of images via the space of patches, that is, square sub-images of fixed size (e.g. 4 × 4). Our main contribution is to consider optimal transport to enforce the multiscale patch distribution of generated images, which leads to two different formulations. First, a pixel-based optimization method is proposed, relying on discrete optimal transport. We show that it is related to a well-known texture optimization framework based on iterated patch nearest-neighbor projections, while avoiding some of its shortcomings. Second, in a semi-discrete setting, we exploit the differential properties of Wasserstein distances to learn a fully convolutional network for texture generation. Once estimated, this network produces realistic and arbitrarily large texture samples in real time. The two formulations result in non-convex concave problems that can be optimized efficiently with convergence properties and improved stability compared to adversarial approaches, without relying on any regularization. By directly dealing with the patch distribution of synthesized images, we also overcome limitations of state-of-the art techniques, such as patch aggregation issues that usually lead to low frequency artifacts (e.g. blurring) in traditional patch-based approaches, or statistical inconsistencies (e.g. color or patterns) in learning approaches.Preprint. Under review.

show abstract

Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture

Cited by 41 publications

References 42 publications

Texture Mixing by Interpolating Deep Statistics via Gaussian Models

Texture Mixing by Interpolating Deep Statistics via Gaussian Models

A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks

Wasserstein Generative Models for Patch-Based Texture Synthesis

Contact Info

Product

Resources

About