FibAR: Embedding Optical Fibers in 3D Printed Objects for Active Markers in Dynamic Projection Mapping

Abstract

Fig. 1. Overview of our proposed system. (a) The internal structure of a projection object. Optical fibers with the same color are connected to the same IR LED attached to the bottom of the object. (b) The projection object fabricated from a multi-material 3D printer. (c) There are seven holes on the bottom of the object, into which IR LEDs are inserted. (d) A captured IR image shows IR light emit from the tip of each fiber, which is transmitted through the fiber from one of the LEDs. (e) A dynamic projection mapping result.Abstract-This paper presents a novel active marker for dynamic projection mapping (PM) that emits a temporal blinking pattern of infrared (IR) light representing its ID. We used a multi-material three dimensional (3D) printer to fabricate a projection object with optical fibers that can guide IR light from LEDs attached on the bottom of the object. The aperture of an optical fiber is typically very small; thus, it is unnoticeable to human observers under projection and can be placed on a strongly curved part of a projection surface. In addition, the working range of our system can be larger than previous marker-based methods as the blinking patterns can theoretically be recognized by a camera placed at a wide range of distances from markers. We propose an automatic marker placement algorithm to spread multiple active markers over the surface of a projection object such that its pose can be robustly estimated using captured images from arbitrary directions. We also propose an optimization framework for determining the routes of the optical fibers in such a way that collisions of the fibers can be avoided while minimizing the loss of light intensity in the fibers. Through experiments conducted using three fabricated objects containing strongly curved surfaces, we confirmed that the proposed method can achieve accurate dynamic PMs in a significantly wide working range.

TsFPS

“…The 6DoF object tracking is an active research area both in computer vision and HCI with numerous applications rely on it. FibAR [26] embeds optical fibers in 3D printed objects as active markers so as to track the object with the corresponding shapes. LightChisel system [6] introduces a cylinder-shaped controller with LEDs inside, two cameras are used for capturing the motions.…”

Multifocal Stereoscopic Projection Mapping

“…Although such a projector can currently be considered special equipment, the one we used in this paper is an off-the-shelf projector that is available in the market [73]. In addition, researchers have found in the last few years that a high-speed projector is essential for realizing immersive dynamic PM applications [20,53,69]. Although our current system applies a gray-scale projector, the manufacturer of the projector just launched a full-color version.…”

Augmenting reality: On the shared history of perceptual illusion and video projection mapping

“…The Ichikawa Senoo Laboratory further demonstrated in 2016 their dynamic projection mapping systems actuating onto a deformed sheet of paper and T shirt on one or multiple targets. Considering the possibility for embedding active or passive markers in clothing and everyday objects [66], these advancements could represent the novel technical feasibility for most surfaces in the real world to become screens, displaying perceptual illusions coherently and seamlessly [67][68][69][70].…”