Real-time video abstraction

Abstract

Figure 1: Abstraction examples. Original: Snapshots of a guard in Petra (left) and two business students (right). Abstracted: After several bilateral filtering passes and with DoG-edges overlayed. Quantized: Luminance channel soft-quantized to 12 bins (left) and 8 bins (right). Note how folds in the clothing and other image details are emphasized (stones on left and student's shadows on right). AbstractWe present an automatic, real-time video and image abstraction framework that abstracts imagery by modifying the contrast of visually important features, namely luminance and color opponency. We reduce contrast in low-contrast regions using an approximation to anisotropic diffusion, and artificially increase contrast in higher contrast regions with difference-of-Gaussian edges. The abstraction step is extensible and allows for artistic or data-driven control. Abstracted images can optionally be stylized using soft color quantization to create cartoon-like effects with good temporal coherence. Our framework design is highly parallel, allowing for a GPU-based, real-time implementation. We evaluate the effectiveness of our abstraction framework with a user-study and find that participants are faster at naming abstracted faces of known persons compared to photographs. Participants are also better at remembering abstracted images of arbitrary scenes in a memory task.

Edge-preserving decompositions for multi-scale tone and detail manipulation

“…Another example is image and video stylization and abstraction [DeCarlo and Santella 2002;Winnemöller et al 2006]. Here the details are discarded, while the base layer is further processed to achieve a stylized look.…”

State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video

“…In doing so, they also drew conclusions with respect to some of Salesin's challenges such as the artistic Turing test by examining techniques that largely did not pass at the time. Evaluation work more closely aligned with Salesin's visual communication challenge was proposed by Gooch et al [43] and Winnemöller et al [164] in their portrait abstractions. It was shown that abstracted faces were more quickly recognized than photorealistic imagery, implying improved comprehension through IB-AR.…”

“…A fully automatic pipeline for the stylization of cartoon renderings based on images and videos was first proposed in the seminal work by Winnemöller et al [164]. Their pipeline employs the bilateral and difference of Gaussians (DoG) filter and contains several influential ideas that other researchers later built upon.…”

“…A schematic overview of a generalized form of the Winnemöller et al [164] pipeline is shown in Fig. 11.…”

“…11. In [70], [82], [164]. After the conversion to CIELab, the input is iteratively abstracted using the bilateral filter.…”