Empathic painting

Abstract

We present the "empathic painting" -an interactive painterly rendering whose appearance adapts in real time to reflect the perceived emotional state of the viewer. The empathic painting is an experiment into the feasibility of using high level control parameters (namely, emotional state) to replace the plethora of low-level constraints users must typically set to affect the output of artistic rendering algorithms. We describe a suite of Computer Vision algorithms capable of recognising users' facial expressions through the detection of facial action units derived from the FACS scheme. Action units are mapped to vectors within a continuous 2D space representing emotional state, from which we in turn derive a continuous mapping to the style parameters of a simple but fast segmentation-based painterly rendering algorithm. The result is a digital canvas capable of smoothly varying its painterly style at approximately 4 frames per second, providing a novel user interactive experience using only commodity hardware.

Reducing affective responses to surgical images through color manipulation and stylization

“…Shugrina et al [2006], for example, presented their "empathic painting" technique that recognizes a person's emotional state based on features of their facial expression, which they then use to adjust the parameters of a painterly rendering technique. Here Shugrina et al borrow from the psychological literature and created a mapping from the detected emotional state to rendering parameters such as stroke path and color.…”

State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video

“…In the parallel SBR branch of semi-automated (i. e., userassisted) algorithms, the low/high-level distinction is again mirrored; with early techniques relying on image filters to orient brush strokes [47] and later work-predating automated measures for emphasis-using gaze trackers to directly harness the perceptual measures inherent in the human visual system [131]. In some recent automated algorithms, stroke placement is influenced by even higher-level contextual parameters such as emotion and mood [22], [141]. Most recently, there has been a trend back toward interaction, producing semi-automated tools for painterly video that enable keyframing of the fields used to arrange strokes [66], [89], [108].…”

“…Segmentation is frequently incorporated as step toward parsing image structure, enabling the adaptation of rendering according to the content in regions. In some techniques, SBR algorithms are applied to render the interiors of regions independently [41], [141], [157]. However, the use of regions as rendering primitives in their own right has also given rise to additional styles including cartoon 'flat' shading [21], [156], new materials such as stained glass [104], [139], felt [109], and even emulation of abstract artistic styles [16].…”

“…Hertzmann's algorithm [55] has influenced many subsequent IB-AR approaches incorporating coarse-tofine rendering [52], [56], [89], [141], [168]. Shiraishi and Yamaguchi [140] also extended the image moment-based painterly technique of Treavett and Chen [151] to operate in multiple passes using windows of decreasing size.…”

“…4.1). Shugrina et al [141] filled region interiors with brush strokes aligned with the principal axis but placed brush strokes on the region boundaries for outlines.…”