Enhancing Color Images of Extremely Low Light Scenes Based on RGB/NIR Images Acquisition With Different Exposure Times

Abstract

We propose a novel method to synthesize a noise- and blur-free color image sequence using near-infrared (NIR) images captured in extremely low light conditions. In extremely low light scenes, heavy noise and motion blur are simultaneously produced in the captured images. Our goal is to enhance the color image sequence of an extremely low light scene. In this paper, we augment the imaging system as well as enhancing the image synthesis scheme. We propose a novel imaging system that can simultaneously capture the red, green, blue (RGB) and the NIR images with different exposure times. An RGB image is taken with a long exposure time to acquire sufficient color information and mitigates the effects of heavy noise. By contrast, the NIR images are captured with a short exposure time to measure the structure of the scenes. Our imaging system using different exposure times allows us to ensure sufficient information to reconstruct a clear color image sequence. Using the captured image pairs, we reconstruct a latent color image sequence using an adaptive smoothness condition based on gradient and color correlations. Our experiments using both synthetic images and real image sequences show that our method outperforms other state-of-the-art methods.

Image Acquisition and Techniques to Perform Image Acquisition

“…The general aim of Image Acquisition is to transform an optical image (Real World Data) into an array of numerical data which could be later manipulated on a computer, before any video or image processing can commence an image must be captured by camera and converted into a manageable entity [4]. The Image Acquisition process consists of three steps:-1.…”

Nonlinear Regression Color Correction Method for RGBN Cameras

“…Yamashita et al [3] separately calculate the integration of the NIR band sensitivity curve and the NIR part of the visible band sensitivity curve to obtain a constant ratio. They corrected the colors by then directly subtracting the visible band and the constant ratio of the NIR band, proposing two optimized versions in the same year [11], [12]. Zahra et al [2] optimized the traditional 3 × 4 CCM by calculating the differences in the sensor's relative spectral sensitivity curve and a designed ideal relative spectral sensitivity curve.…”

Convolutional Neural Network Training for RGBN Camera Color Restoration Using Generated Image Pairs

“…The method is limited by the measurement accuracy of the sensor response. Yamashita [12][13][14] measured the response curve of each channel and calculated the ratio of the integral of the NIR channel curve to the integrals of each of the color channel curves, then directly subtracted the NIR channel from the color channels with fixed ratios that assumed these ratios would not change as the scene changed. Aguilera et al [15] proposed a learning-based RGBN camera color restoration method with a simple network structure composed of two fully connected layers.…”