Estimating Illumination Direction for Augmented Reality in Real-Time by using Low-Cost Sensors

Abstract

In recent years, Augmented Reality has become a very popular topic, both as a research and commercial field. This trend has originated with the use of mobile devices as computational core and display. The appearance of virtual objects and their interaction with the real world is a key element in the success of an Augmented Reality software. A common issue in this type of software is the visual inconsistency between the virtual and real objects due to wrong illumination. Although illumination is a common research topic in Computer Graphics, few studies have been made about real time estimation of illumination direction. In this work we present a low-cost approach to detect the direction of the environment illumination, allowing the illumination of virtual objects according to the real light of the ambient, improving the integration of the scene. Our solution is open-source, based on Arduino hardware and the presented system was developed on Android.

Real-time Illumination and Visual Coherence for Photorealistic Augmented/Mixed Reality

“…Therefore, to address these challenges, multiple methods have been suggested to overcome the problem and produce acceptable results. Some successful methods used high dynamic range (HDR-video) cameras in real-time attached to typical light probes and typical video cameras with distinctive optical filters or customized light probes [54,61,89,96,99,106].…”

“…The light source of a large area such as the sky can be a secondary approximation. Shadow cues can be used in the image for enhanced outcomes [10,65,66,99,109].…”

“…The representation of a shadow can be computed in the first pass, while this representation could be used for the shading of the surface points of the final image in the second pass. The existent shadowing techniques are influenced by shadow volumes and shadow mapping [24,42,88,99].…”

“…There are two passes for shadow mapping technique: in the first pass, the whole scene is rendered from the perspective of the light source to the depth buffer; in the second pass, the shadow map is used to determine if a fragment is occluded also from the light source perspective to be rendered from the viewpoint of an observer. The fragment is considered part of the shadow if its depth in the coordinate of the light source is higher than the shadow map entry [24,54,74,99].…”

“…In the current studies, the blending approach is applied to create soft shadows that can be pre-calculated even in the dynamic illumination scenes [74,99].…”