AR marker hiding based on image inpainting and reflection of illumination changes

Abstract

This paper proposes a new method of diminished reality which removes AR markers from a user's view image in real time. To achieve natural marker hiding, assuming that an area around a marker is locally planar, the marker area in the first frame is inpainted using the rectified image to achieve high-quality inpainting. The unique inpainted texture is overlaid on the marker region in each frame according to camera motion for geometric consistency. Both global and local luminance changes around the marker are separately detected and reflected to the inpainted texture for photometric consistency.

Marker Hiding Methods: Applications in Augmented Reality

“…In 2012, a different method was proposed (Kawai 2012) that was based on the diminished reality technique that uses image inpainting to generate textures on objects rather than using a real background texture to over-lay the object regions. In this method, both global and local illumination changes are reflected to the generated texture that hides the marker area.…”

“…The marker area for texture generation (adapted from Siltanen 2006). (Korkalo, Aittala, and Siltanen 2010;Kawai et al 2012). Moreover, to overcome the varying lighting conditions, the generated texture is partly transparent near the edges, and this transparent part covers the real back-ground outside the marker, which is the area between the white border of the marker and the yellow square in Figure 4.…”

“…A new method was proposed that focuses on the diminished reality technique, which generates textures on objects by image inpainting instead of overlaying a real background texture onto the object regions (Kawai et al 2013). This method has been used to hide the augmented reality marker based on the previous work in Kawai et al (2012), which uses the exemplar-based inpainting method to inpaint the marker region. In general, the exemplar-based inpainting method can combine the linear structure reconstruction and texture synthesis for higher quality and enhanced result of the inpainted area.…”

User-Aware Audio Marker Using Low Frequency Ultrasonic Object Detection and Communication for Augmented Reality

“…Augmented reality (AR) is usually described as the interaction between a real environment and computer-generated virtual objects through allowing users to interact with the virtual objects through visual, auditory, or haptic sensory modalities [1,2]. Distinct visual cues, such as image markers [1][2][3][4] or parts of the human body [5], are commonly used to place virtual objects into the real world. Among the various AR services, there are indoor guidance services widely used in public places such as museums and airports [1][2][3].…”

High-Quality Real-Time Video Inpaintingwith PixMix

“…Some approaches were introduced allowing for marker hiding in AR applications by semi-dynamic textures combined with alpha blending [14], [15]. Other approaches realize Diminished Reality by combining images of several cameras, observing a certain area from different viewpoints [16], [17].…”