Novel Software-Based Method to Widen Dynamic Range of CCD Sensor Images

Abstract

Abstract. In the past twenty years, CCD sensor technology has made significant progress in increasing resolution and improving low-light performance by hardware. However due to physical limits of the sensor design and fabrication, fill factor has become the bottle neck problem for improving quantum efficiency of CCD sensor in order to widen dynamic range of output image. In this paper we propose a novel software-based method to compensate the performance degradation of the dynamic range, by virtual increase of fill factor which is achieved by a resampling process. In our method, the CCD sensor images are rearranged to a new grid of virtual sensor pixels, each of which is composed by subpixels. A statistical framework consisting of local learning model, simulations and Bayesian inference is used to estimate new subpixel intensity values. The highest probability of sub-pixels intensity values in each resampled pixel area is used to estimate the pixel intensity values of a new dynamic range enhanced image.From generation of gray level optical images, CCD images with different fill factors were obtained. By knowing the fill factor and having the CCD image, a new resampled image was computed. Each resampled image was compared to the respective CCD and optical image. The results of such comparison show that by using the proposed method it is possible to widen significantly the recordable dynamic range of CCD images and obtain a virtual increase of fill factor to 100%. The stability and fill factor dependency of the proposed method were also examined in which the results showed insignificant spreading effect and fill factor dependency.

Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image

“…As the exposure time for capturing an image is fixed for all scene points and the fill factor of a camera is generally far from 100% [14], we should surely assume that only part of the dynamic range of L is captured. The nonlinear function of $f$, which is caused by components in IAP and IPP, maps the image irradiance Z to the output of IAP; the IAP image $M$, as: $M_i=f\left(Z_i\right)=f\left(E_i\tau \zeta \right)$ by assuming that the sensor output increases monotonically, or at least semi-monotonically, with respect to $Z_i$, the function $f$ is invertible.…”

“…This solution is used in the recovery of high dynamic ranges of images [30,31]. Another solution to the problem expressed in Equation (2) is to use a single captured image and vary the fill factors to obtain a set of images; i.e., the exposure time remains the same and the images are generated virtually [14]. Accordingly, this time Equation (2) becomes: $g_v\left(M{v}_{ij}\right)=f^{-1}\left(M{v}_{ij}\right)=\tau E_i{\zeta }_j,$ where $g_v$ is the virtual camera response function, $j$ is the fill factor index, and $Mv$ is the generated virtual image.…”

“…In our work a 32 × 32 patch of pixels with rich variation of pixel intensities is selected automatically; i.e., the range of pixel intensity variations of all possible 32 × 32 patches are examined by calculating the number of tonal levels in their histograms to select the patch. Then the patch is normalized by removing the offset of its histogram from zero, and a set of $N$ virtual images is generated by the proposed method in [14], assuming knowing a fill factor from a set of fill factors between 22% and 81%. According to the method, the process of virtual image generation is divided into three steps of (a) projecting the pixel intensity onto a new grid of subpixels; (b) estimating the values of subpixels based on a local learning model; and (c) estimating the new pixel intensity by decision-making.…”

“…At the same time, compared to human eyes that have a much larger range of luminance (10,000:1 vs. 100:1 cd/m 2 ) [13], it is necessary for a camera sensor to have a larger fill factor with a die size to create more intensity levels for an image in the luminance range. Recently a new approach was presented [14] in which the fill factor is increased virtually, resulting in a significant extension of image tonal level and widening of the dynamic range. In the approach, the original fill factor is assumed to be known.…”

“…In our method, from the captured image, a set of $N$ new images with $N$ number of fill factors are generated based on the proposed method in [14]. Then the virtual response function of the imaging process is estimated from the set of generated images which combines the images into a composite virtual radiance map.…”