Foreground and shadow occlusion handling for outdoor augmented reality

Abstract

Figure 1: Foreground and shadow occlusions are handled correctly with our proposed solution. The two images on the left shows the original frame in campus sequences and its corresponding augmented result. The two ones on the right shows the original frame in Asuka sequences and its corresponding result. ABSTRACTOcclusion handling in augmented reality (AR) applications is challenging in synthesizing virtual objects correctly into the real scene with respect to existing foregrounds and shadows. Furthermore, outdoor environment makes the task more difficult due to the unpredictable illumination changes. This paper proposes novel outdoor illumination constraints for resolving the foreground occlusion problem in outdoor environment. The constraints can be also integrated into a probabilistic model of multiple cues for a better segmentation of the foreground. In addition, we introduce an effective method to resolve the shadow occlusion problem by using shadow detection and recasting with a spherical vision camera. We have applied the system in our digital cultural heritage project named Virtual Asuka (VA) and verified the effectiveness of the system.

Shadow Generation in Mixed Reality: A Comprehensive Survey

“…Virtual Asuka [42] is an AR project where shadow mapping is applied and image-based algorithms for shadow detection and recasting with a spherical vision camera are employed. First, shadow regions are detected using camera sensitivity.…”

“…First, shadow regions are detected using camera sensitivity. Then, by applying the illumination invariant constraint and employing the energy minimization method [42] the shadow regions are picked up and used to recast shadow onto the virtual object with the spherical vision camera. Recasting the shadow regions on the virtual objects forms the main part of this algorithm.…”

Outdoor gallery and its photometric issues

“…Therefore, it is wise to combine different available cues together to segment the foreground online because a single cue is not reliable. We extend the work of Kakuta et al [Kakuta et al 2008] by adding the illumination cue and the motion cue with background attenuation which are explained in detail in [Lu et al 2010]. …”

“…Finally, the optimum label can also be optimized using energy minimization with graph cut as explained in [Lu et al 2010]. …”

“…If we assume that the camera sensitivity is sufficiently narrow and that daylight is blackbody radiation, we can easily infer the constraint among the log-ratios of image irradiances ln R λ R t 1 ,t 2 , ln R λ G t 1 ,t 2 and ln R λ B t 1 ,t 2 as in[Lu et al 2010]:d c (I 1 , I 2 ) = ∥ ln R λ R t 1 ,t 2 −C ln R λ G t 1 ,t 2 + (C − 1) ln R λ B t 1 ,t 2 ∥ = 0(2)…”