Occlusion Handling Method for Ubiquitous Augmented Reality Using Reality Capture Technology and GLSL

Abstract

ABSTRACT:The primary challenge in generating convincing Augmented Reality (AR) graphics is to project 3D models onto a user's view of the real world and create a temporal and spatial sustained illusion that the virtual and real objects co-exist. Regardless of the spatial relationship between the real and virtual objects, traditional AR graphical engines break the illusion of co-existence by displaying the real world merely as a background, and superimposing virtual objects on the foreground. This research proposes a robust depth sensing and frame buffer algorithm for handling occlusion problems in ubiquitous AR applications. A high-accuracy Time-of-flight (TOF) camera is used to capture the depth map of the real-world in real time. The distance information is processed using the OpenGL Shading Language (GLSL) and rendered into the graphics depth buffer, allowing accurate depth resolution and hidden surface removal in composite AR scenes. The designed algorithm is validated in several indoor and outdoor experiments using the SMART AR framework.

Occlusion Handling in Video-Based Augmented Reality Using the Kinect Sensor for Indoor Registration

“…Dong et al [1] propose an algorithm for occlusion handling using depth obtained by a high-resolution TOF camera (PMD CamCube 3.0) and technology based on hardware to supress the background illumination. The authors add a second camera in order to obtain a RGB image; the first task performed is the alignment of both images.…”

“…According to some of the most recent works in the literature [3], [1], the use of depth information about the real world has lead to better results in the occlusion handling in AR. In this approach several stereo vision systems and 3D cameras have been used to calculate the distances in the real world.…”