Augmenting physical avatars using projector-based illumination

Abstract

Figure 1: Our system allows augmentation of a physical avatar (a) with projector-based illumination, significantly increasing its expressiveness. In (b) the target performance is shown. The appearance under controlled and ambient illumination is shown in (c) and (d). AbstractAnimated animatronic figures are a unique way to give physical presence to a character. However, their movement and expressions are often limited due to mechanical constraints. In this paper, we propose a complete process for augmenting physical avatars using projector-based illumination, significantly increasing their expressiveness. Given an input animation, the system decomposes the motion into low-frequency motion that can be physically reproduced by the animatronic head and high-frequency details that are added using projected shading. At the core is a spatio-temporal optimization process that compresses the motion in gradient space, ensuring faithful motion replay while respecting the physical limitations of the system. We also propose a complete multi-camera and projection system, including a novel defocused projection and subsurface scattering compensation scheme. The result of our system is a highly expressive physical avatar that features facial details and motion otherwise unattainable due to physical constraints.

Adaptive stray-light compensation in dynamic multi-projection mapping

“…We show that, with a simplifying assumption, this is not required. As in our light transport computations, Bermano et al [7] solved the contribution from multiple projectors, while also accounting for subsurface scattering and projector defocus. However, their system does not compute the results in real time.…”

Deformation Lamps

“…Bimber and Iwai [12] proposed a light projection method to enhance the luminance contrast and color of printed materials, and Bimber et al and later studies have developed algorithms to modify the appearance of real objects by light projection with image compensation [4,14]. The technique is used to edit the appearance of real objects [2,3], and moreover, is also employed to add motion impressions to a static object by projecting a moving pattern [10,44].…”

“…For example, a previous study [44] demonstrated a technique that indirectly adds motion impressions to a static object by projecting a moving pattern to the background and generating a visual illusion called 'induced motion' [21]. A recent study has developed projector-based illumination with precise image compensation for production of high-frequency details of dynamic facial expressions of physical avatars [10].…”

ChameleonMask

“…Projecting enables a high level of expression, such as facial details and motion [1]. However, these techniques do not suitably apply to wearable systems due to their lack of projector placement.…”

“…In addition, local users also have a more realistic sensation of the remote user's presence when an autonomous robot is used [12]. 1 Double, http://www.doublerobotics.com/business/ 2 Should I Attend a Conference Via a Telepresence Robot?, http://spectrum.ieee.org/automaton/robotics/industrialrobots/attending-conference-via-telepresence-robot 3 The reason why I could attend in conference from 200km far way with a child first 2weeks after delivery. (in Japanese), http://bylines.news.yahoo.co.jp/yukiigarashi/20141211-00041418/ However, these systems are still imperfect.…”