Texture optimization for example-based synthesis

Kwatra, Vivek; Essa, Irfan; Bobick, Aaron F.; Kwatra, Nipun

doi:10.1145/1073204.1073263

Cited by 537 publications

(425 citation statements)

References 23 publications

Supporting

Mentioning

421

Contrasting

Unclassified

Order By: Relevance

“…We minimize a patch-based functional taken from the video inpainting [19,20] and texture synthesis [13] literature. Although the core of our algorithm is similar to that of Arias et al, [9] there are several important differences, which we list here.…”

Section: Introductionmentioning

confidence: 99%

Non-Local Patch-Based Image Inpainting

Newson¹,

Almansa²,

Gousseau³

et al. 2017

Image Processing on Line

View full text Add to dashboard Cite

Image inpainting is the process of filling in missing regions in an image in a plausible way. In this contribution, we propose and describe an implementation of a patch-based image inpainting algorithm. The method is actually a two-dimensional version of our video inpainting algorithm The functional specifies that a good solution to the inpainting problem should be an image where each patch is very similar to its nearest neighbor in the unoccluded area. Iterations are performed in a multi-scale framework which yields globally coherent results. In this manner two of the major goals of image inpainting, the correct reconstruction of textures and structures, are addressed. We address a series of important practical issues which arise when using such an approach. In particular, we reduce execution times by using the PatchMatch [C. Barnes, PatchMatch: a randomized correspondence algorithm for structural image editing, ACM Transactions on Graphics, (2009)] algorithm for nearest neighbor searches, and we propose a modified patch distance which improves the comparison of textured patches. We address the crucial issue of initialization and the choice of the number of pyramid levels, two points which are rarely discussed in such approaches. We provide several examples which illustrate the advantages of our algorithm, and compare our results with those of state-of-the-art methods. Source CodeThe reviewed source code and documentation for this algorithm are available from the web page of this article 1 .

show abstract

Section: Introductionmentioning

confidence: 99%

Non-Local Patch-Based Image Inpainting

Newson¹,

Almansa²,

Gousseau³

et al. 2017

Image Processing on Line

View full text Add to dashboard Cite

show abstract

“…However, for patch-based methods it is difficult to make the pixels' appearance change at pixel level. The optimizationmethod [10] adopts the energy minimization function to synthesize the texture as a whole, which usually produces improved synthesis results. In the recent years, researchers further proposed methods dedicated to images with structured objects which preserve the structures during synthesis [11][12][13].…”

Section: Related Workmentioning

confidence: 99%

An improved image analogy method based on adaptive CUDA-accelerated neighborhood matching framework

et al. 2012

View full text Add to dashboard Cite

The image analogy framework is especially useful to synthesize appealing images for non-homogeneous input and gives users creative control over the synthesized results. However, the traditional framework did not adaptively employ the searching strategy based on neighborhood's different textural contents. Besides, the synthesis speed is slow due to intensive computation involved in neighborhood matching. In this paper we present a CUDA-based neighborhood matching algorithm for image analogy. Our algorithm adaptively applies the global search of the exact L 2 nearest neighbor and k-coherence search strategies during synthesis according to different textural features of images, which is especially usefully for non-homogeneous textures. To consistently implement the above two search strategies on GPU, we adopt the fast k nearest neighbor searching algorithm based on CUDA. Such an acceleration greatly reduces the time of the pre-process of k-coherence search and the synthesis procedure of the global search, which makes possible the adjustment of important synthesis parameters. We further adopt synthesis magnification to get the final high-resolution synthesis image for running efficiency. Experimental results show that our algorithm is suitable for various applications of the image analogy framework and takes full advantage of GPU's parallel processing capability to improve synthesis speed and get satisfactory synthesis results.

show abstract

“…Given a small texture input, texture synthesis techniques [3,9,22] automatically generate seamless textures over a large surface. In recent years, researchers have designed the GPU texture synthesis algorithms to render the large synthesized results in real-time with the small input sample stored in the texture memory [11,12,20].…”

Section: Previous Workmentioning

confidence: 99%

“…We also compute the coordinate difference, which gives the relative position of the current texel within a block (line 7). We then fetch transformation coefficients from two transformation textures, including 6 affine coefficients and a codebook type flag (line [8][9]. According to these coefficients, we prepare the codebook texture coordinate (line [11][12] and then read the texture color from public or private codebook based on the flag taffine1.w (line [13][14][15].…”

Section: Texture Decompressionmentioning

confidence: 99%

Compressing repeated content within large-scale remote sensing images

et al. 2012

View full text Add to dashboard Cite

Large-scale remote sensing images, including both satellite and aerial photographs, are widely used to render terrain scenes in real-time geographic visualization systems. Such systems often require large memories in order to store fine terrain details and fast network speeds to transfer image data, if they are built as web applications. In this paper, we propose a progressive texture compression framework to reduce the memory and bandwidth cost by compressing repeated content within and among large-scale remote sensing images. Different from existing image factorization methods, our algorithm incrementally find similar regions in new images so that large-scale images can be more efficiently compressed over time. We further propose a descriptor, the Gray Split Rotate (GSR) descriptor, to accelerate the similarity search. The reconstruction quality is finally improved by compressing residual error maps using customized S3TC-like compression. Our experiment shows that even with the error maps, our system still has higher compression rate and higher compression quality than using S3TC alone, which is a typical compression solution in most existing visualization systems.

show abstract

Texture optimization for example-based synthesis

Cited by 537 publications

References 23 publications

Non-Local Patch-Based Image Inpainting

Non-Local Patch-Based Image Inpainting

An improved image analogy method based on adaptive CUDA-accelerated neighborhood matching framework

Compressing repeated content within large-scale remote sensing images

Contact Info

Product

Resources

About