Automatic Design of Color Filter Arrays in The Frequency Domain

Abstract

In digital color imaging, the raw image is typically obtained through a single sensor covered by a color filter array (CFA), which allows only one color component to be measured at each pixel. The procedure to reconstruct a full color image from the raw image is known as demosaicking. Since the CFA may cause irreversible visual artifacts, the CFA and the demosaicking algorithm are crucial to the quality of demosaicked images. Fortunately, the design of CFAs in the frequency domain provides a theoretical approach to handling this issue. However, almost all the existing design methods in the frequency domain involve considerable human effort. In this paper, we present a new method to automatically design CFAs in the frequency domain. Our method is based on the frequency structure representation of mosaicked images. We utilize a multi-objective optimization approach to propose frequency structure candidates, in which the overlap among the frequency components of images mosaicked with the CFA is minimized. Then, we optimize parameters for each candidate, which is formulated as a constrained optimization problem. We use the alternating direction method to solve it. Our parameter optimization method is applicable to arbitrary frequency structures, including those with conjugate replicas of chrominance components. Experiments on benchmark images confirm the advantage of the proposed method.

Deep Joint Design of Color Filter Arrays and Demosaicing

“…Table compares the PSNR of the results obtained with our 4 × 4 noise‐free CFA (Fig. ) with the ones produced by the most successful demosaicing techniques [MBP*09, Get11b, BLLY16, HLLD11, Con11, Wan14, Cha16, GCPD16]. For all comparisons, we have used either source or executable code provided by the authors.…”

“…But the masks can be used to impose additional constraints. For instance, one can use them to find 2 × 2 patterns with only three colors (as in the Bayer pattern), or to enforce designs that follow specific patterns, such as blue‐noise characteristics [Con10], or diagonal designs [LP05, BLLY16].…”

“…Hao et al [HLLD11] introduced CFAs based on the frequency structure, which are manually initialized according to some guidelines, and then optimized using a geometric method. Bai et al [BLLY16] proposed a method for CFA design in the frequency domain. Given the CFA pattern size, their method suggests frequency‐structure candidates and then optimizes the parameters by maximizing numeric stability of color transformations.…”

“…The Bayer pattern [Bay76] is the most popular CFA pattern for digital cameras and many demosaicing algorithms have been proposed to improve the quality of the reconstructed images sampled with it [MBP*09, Get11b,HvLP12,Wan14]. Nonetheless, several techniques have focused on designing new CFA patterns, either by looking for patterns whose representation in the frequency domain have little overlap between luma and chroma information [HLLD11,Con11,BLLY16] (designed specifically for use with frequency‐selection demosaicing algorithms [ASH05,Dub05]), or by designing CFAs for sparse‐representation‐based demosaicings [LBLY17, MBP*09]. Essentially, previous solutions have been restricted to either design demosaicing algorithms for a given CFA pattern ( e.g ., the Bayer pattern), or to design CFA patterns that work with a specific demosaicing algorithm ( e.g ., [ASH05, Dub05, MBP*09]).…”

Random Color Filter Arrays are Better than Regular Ones

“…[13][14][15] Many authors have proposed optimal CFAs arrangement based on the criteria of frequency representation and selection. [16][17][18][19][20] Indeed, the mosaic arrangement of the filters in the CFA could be interpreted as a spatial multiplexing of color components and has a simple expression in the Fourier domain. [21][22][23] The spatial Fourier representation of the CFA allows simple linear demosaicing by selecting the part of the spectrum that corresponds to luminance and color components.…”