Stylizing animation by example

Bénard, Pierre; Cole, Forrester; Kass, Michael; Mordatch, Igor; Hegarty, James; Senn, Martin Sebastian; Fleischer, Kurt; Pesare, Davide; Breeden, Katherine

doi:10.1145/2461912.2461929

Cited by 72 publications

(66 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Patch-based synthesis can mimic the style of artists, such as oil paint style, watercolor style, or even abstract style. We discuss an early work in this area, image analogies [66], followed by two recent works, Bénard et al [67] and StyLit, which perform example-based stylization using patches. The image analogies framework [66] gave early results showing the transfer of oil paint and watercolor styles.…”

Section: Synthesis For Specialized Domainsmentioning

confidence: 99%

See 1 more Smart Citation

A survey of the state-of-the-art in patch-based synthesis

Barnes

Zhang

2017

Comp. Visual Media

View full text Add to dashboard Cite

This paper surveys the state-of-the-art of research in patch-based synthesis. Patch-based methods synthesize output images by copying small regions from exemplar imagery. This line of research originated from an area called "texture synthesis", which focused on creating regular or semi-regular textures from small exemplars. However, more recently, much research has focused on synthesis of larger and more diverse imagery, such as photos, photo collections, videos, and light fields. Additionally, recent research has focused on customizing the synthesis process for particular problem domains, such as synthesizing artistic or decorative brushes, synthesis of rich materials, and synthesis for 3D fabrication. This report investigates recent papers that follow these themes, with a particular emphasis on papers published since 2009, when the last survey in this area was published. This survey can serve as a tutorial for readers who are not yet familiar with these topics, as well as provide comparisons between these papers, and highlight some open problems in this area.

show abstract

Section: Synthesis For Specialized Domainsmentioning

confidence: 99%

“…[68] to improve results for such exemplar-based stylization. The method of Bénard et al [67] allows artists to paint over certain keyframes in a 3D rendered animation. Other frames are automatically stylized in the target style by using a temporally coherent extension of image analogies to the video cube.…”

Section: Synthesis For Specialized Domainsmentioning

confidence: 99%

A survey of the state-of-the-art in patch-based synthesis

Barnes

Zhang

2017

Comp. Visual Media

View full text Add to dashboard Cite

show abstract

“…Image analogies permit many effects including transfer of painterly styles by example. By-example video stylization was explored by Hashimoto et al [2003], Bousseau et al [2007], and Bénard et al [2013]. High-resolution was precluded in those works by computational cost.…”

Section: Related Workmentioning

confidence: 99%

“…Rather, this application demonstrates the efficiency of our data structure. In particular, if slower run-time is acceptable, one could improve quality by using the backwards advection of Bousseau et al [2007] or the video cube optimization of Bénard et al [2013].…”

Section: Artistic Video Stylizationmentioning

confidence: 99%

PatchTable

et al. 2015

View full text Add to dashboard Cite

[Hertzmann et al. 2001] AbstractThis paper presents a data structure that reduces approximate nearest neighbor query times for image patches in large datasets. Previous work in texture synthesis has demonstrated real-time synthesis from small exemplar textures. However, high performance has proved elusive for modern patch-based optimization techniques which frequently use many exemplar images in the tens of megapixels or above. Our new algorithm, PatchTable, offloads as much of the computation as possible to a pre-computation stage that takes modest time, so patch queries can be as efficient as possible. There are three key insights behind our algorithm: (1) a lookup table similar to locality sensitive hashing can be precomputed, and used to seed sufficiently good initial patch correspondences during querying, (2) missing entries in the table can be filled during precomputation with our fast Voronoi transform, and (3) the initially seeded correspondences can be improved with a precomputed knearest neighbors mapping. We show experimentally that this accelerates the patch query operation by up to 9× over k-coherence, up to 12× over TreeCANN, and up to 200× over PatchMatch. Our fast algorithm allows us to explore efficient and practical imaging and computational photography applications. We show results for artistic video stylization, light field super-resolution, and multiimage editing.

show abstract

“…Bousseau et al [2007] address the problem of coherent advection of watercolor stroke texture, also following optical flow for stylizing video. Recent work by Bénard et al [2013] generates example-based stylizations of 3D computer-generated animation by extending the image analogies method of Hertzmann et al [2001] to provide temporal coherence. Our approach is similar to these prior methods in that drawn shapes are advected to follow the motion of an underlying fluid simulation (rather than optical flow in video or 3D animation).…”

Section: Related Workmentioning

confidence: 99%

Stylized keyframe animation of fluid simulations

Browning

Barnes

Ritter

et al. 2014

Proceedings of the Workshop on Non-Photorealistic Animation and Rendering

View full text Add to dashboard Cite

simulation 45 keyframe 50 in-between 53 in-between 57 keyframe 60 in-between 65 keyframe 70Figure 1: An upward-flowing jet of ink rendered in oil pastel. An underlying simulation is used as input to guide the artwork (frame 45 is shown). Frames 50, 60 and 70 are hand-drawn keyframes, while the remaining in-betweens were generated automatically by our system. AbstractWe present a method that combines hand-drawn artwork with fluid simulations to produce animated fluids in the visual style of the artwork. Given a fluid simulation and a set of keyframes rendered by the artist in any medium, our system produces a set of in-betweens that visually matches the style of the keyframes and roughly follows the motion from the underlying simulation. Our method leverages recent advances in patch-based regenerative morphing and image melding to produce temporally coherent sequences with visual fidelity to the target medium. Because direct application of these methods results in motion that is generally not fluid-like, we adapt them to produce motion closely matching that of the underlying simulation. The resulting animation is visually and temporally coherent, stylistically consistent with the given keyframes, and approximately matches the motion from the simulation. We demonstrate the method with animations in a variety of visual styles.

show abstract

Stylizing animation by example

Cited by 72 publications

References 50 publications

A survey of the state-of-the-art in patch-based synthesis

A survey of the state-of-the-art in patch-based synthesis

PatchTable

Stylized keyframe animation of fluid simulations

Contact Info

Product

Resources

About