WISE: Whitebox Image Stylization by Example-Based Learning

Lötzsch, Winfried; Reimann, Max; Martin, Büssemeyer,; Semmo, Amir; Döllner, Jürgen; Trapp, Matthias

doi:10.1007/978-3-031-19790-1_9

Cited by 3 publications

(1 citation statement)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, Lötzsch et al . [LRB*22] use differentiable image filters, a collection of manually designed, parameterized filters, to stylize the input image. As this method does not rely on neural network inference, it is also applicable to high‐resolution input images.…”

Section: Related Workmentioning

confidence: 99%

Stylize My Wrinkles: Bridging the Gap from Simulation to Reality

Weiss,

Stanhope,

Chandran

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

Modeling realistic human skin with pores and wrinkles down to the milli‐ and micrometer resolution is a challenging task. Prior work showed that such micro geometry can be efficiently generated through simulation methods, or in specialized cases via 3D scanning of real skin. Simulation methods allow to highly customize the wrinkles on the face, but can lead to a synthetic look. Scanning methods can lead to a more organic look for the micro details, however these methods are only applicable to small skin patches due to the required image resolution. In this work we aim to overcome the gap between synthetic simulation and real skin scanning, by proposing a method that can be applied to large skin regions (e.g. an entire face) with the controllability of simulation and the organic look of real micro details. Our method is based on style transfer at its core, where we use scanned displacement maps of real skin patches as style images and displacement maps from an artist‐friendly simulation method as content images. We build a library of displacement maps as style images by employing a simplified scanning setup that can capture high‐resolution patches of real skin. To create the content component for the style transfer and to facilitate parameter‐tuning for the simulation, we design a library of preset parameter values depicting different skin types, and present a new method to fit the simulation parameters to scanned skin patches. This allows fully‐automatic parameter generation, interpolation and stylization across entire faces. We evaluate our method by generating realistic skin micro details for various subjects of different ages and genders, and demonstrate that our approach achieves a more organic and natural look than simulation alone.

show abstract

Section: Related Workmentioning

confidence: 99%

Stylize My Wrinkles: Bridging the Gap from Simulation to Reality

Weiss,

Stanhope,

Chandran

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

Artistic style decomposition for texture and shape editing

Reimann,

Büßemeyer,

Buchheim

et al. 2024

Vis Comput

View full text Add to dashboard Cite

While methods for generative image synthesis and example-based stylization produce impressive results, their black-box style representation intertwines shape, texture, and color aspects, limiting precise stylistic control and editing of artistic images. We introduce a novel method for decomposing the style of an artistic image that enables interactive geometric shape abstraction and texture control. We spatially decompose the input image into geometric shapes and an overlaying parametric texture representation, facilitating independent manipulation of color and texture. The parameters in this texture representation, comprising the image’s high-frequency details, control painterly attributes in a series of differentiable stylization filters. Shape decomposition is achieved using either segmentation or stroke-based neural rendering techniques. We demonstrate that our shape and texture decoupling enables diverse stylistic edits, including adjustments in shape, stroke, and painterly attributes such as contours and surface relief. Moreover, we demonstrate shape and texture style transfer in the parametric space using both reference images and text prompts and accelerate these by training networks for single- and arbitrary-style parameter prediction.

show abstract