Real Time Light Source Estimation Using a Fish-Eye Lens with ND Filters

Abstract

3D virtual objects can be augmented in the streaming data through a webcam for various purposes using a marker and AR techniques. But the augmented results without shadows it can be shown unnaturally because the illumination of the real scene does not be considered. To minimize the unnaturalness between the background and virtual object, the illumination of the background is uniformly applied to the virtual object and real object. In this paper we propose a method to estimate light source information in real time for realistic synthesis. The proposed estimation system consists of a camera, fish-eye lens and ND (Neutral Density) filters. The fish-eye lens covers 180 degree of the real environment. Using these hemi-sphere images through the lens the direction of the lights can be estimated. As it considers the energy of pixels in the images the bright regions contain more light information than the other regions. To detect these regions image processing is additionally needed. But using iris of fish-eye lens and ND filters which reduce the incident light we can estimate the bright regions without additional image processing. The proposed method can estimate the light source information in real time and it can be used for an AR or VR environment to increase the naturalness.

A Lighting Consistency Technique for Outdoor Augmented Reality Systems Based on Multi-Source Geo-Information

“…In these applications, identifying how to estimate lighting conditions in the outdoor environment to achieve good lighting consistency is a crucial issue for improving user immersion. Currently, mainstream research methods for this problem can be divided into three categories: methods based on marker information [10][11][12], methods based on auxiliary equipment [13][14][15][16], and methods based on image analysis [17][18][19][20]. Methods based on marker information and methods based on auxiliary equipment require special devices that collect prior information, such as light probes [10] or fisheye cameras [13], making them difficult to implement on mobile devices.…”

“…Currently, mainstream research methods for this problem can be divided into three categories: methods based on marker information [10][11][12], methods based on auxiliary equipment [13][14][15][16], and methods based on image analysis [17][18][19][20]. Methods based on marker information and methods based on auxiliary equipment require special devices that collect prior information, such as light probes [10] or fisheye cameras [13], making them difficult to implement on mobile devices. Methods based on image analysis are less restricted by application scenarios but may not accurately estimate lighting conditions in low-information outdoor environments [17][18][19].…”

“…In order to obtain lighting information from a scene without the need for markers or prior information, techniques have been developed that use auxiliary devices for lighting estimation [13][14][15][16]. Yoo et al [13] and Pardel et al [14] used fisheye lenses and stereo cameras, respectively, to capture HDR panoramic images as environment maps, thereby obtaining lighting information from the scene for real-time rendering. Gruber et al [15] reconstructed a scene model using a depth camera and estimated the lighting of the model using spherical harmonic functions, enabling the generation of soft shadows for virtual objects.…”

Improved Variance Cut Algorithm for Light Source Estimation in Augmented Reality

“…By Using a calibration object without distance illumination assumption [11] The techniques above require a high computational complexity and sometimes become difficult for real time illumination estimation. The previous sampling algorithm for light source estimation described in [12] [13] reduces the computational complexity but at the same time the efficiency of algorithm reduces with different intensity regions.…”

Light Source Estimation in Mobile Augmented Reality Scenes by Using Human Face Geometry

“…Yoo et al [12] capture light information by using a 185 degree fish-eye lens and Neutral density filter (NDF) to have no changes in hue of color rendition by modifying the intensity of all wavelengths or colors of light equally. They stated that lighting maps can be extracted precisely by using fish eye lenses with NDF.…”