Geometrically-Correct Projection-Based Texture Mapping onto a Deformable Object

Abstract

Abstract-Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object's surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

Robust Reflectance Estimation for Projection-Based Appearance Control in a Dynamic Light Environment

“…When the projector-camera system consists of a single camera and single projector, the reflectance can be estimated using (9). However, when the system incorporates two cameras, we must also consider their geometrical relationship and color spaces.…”

“…Many studies have addressed projector-camera systems that are capable of interactively controlling a projection with the changes in the object onto which an image is being projected. For example, there are projection technologies that can project light onto a static object with a complex geometry [7], a face [8], and other deformable objects [9], [10]. Some of these interactive systems first capture the original colors and textures of the target objects and then artificially change their appearance by overlaying a projection.…”

Recent Advances in Projection Mapping Algorithms, Hardware and Applications

“…A high‐speed camera is used to robustly track dot cluster markers drawn by the same invisible inks [NWI15]. Alternatively, retro‐reflective markers are used to measure the surface deformation in the word of Fujimoto et al [FST*14]. However, a fully dynamic tracking is not achieved by this method.…”

Light Projection-Induced Illusion for Controlling Object Color

“…Their system projects light not only onto the walls of the room, but also the players to create an immersive experience. Similarly, Fujimoto et al [4] manipulate the appearance of a deformable cloth by projecting the desired pattern onto it to assist designers in evaluating their work.…”