In digital color imaging, the raw image is typically obtained through a single sensor covered by a color filter array (CFA), which allows only one color component to be measured at each pixel. The procedure to reconstruct a full color image from the raw image is known as demosaicking. Since the CFA may cause irreversible visual artifacts, the CFA and the demosaicking algorithm are crucial to the quality of demosaicked images. Fortunately, the design of CFAs in the frequency domain provides a theoretical approach to handling this issue. However, almost all the existing design methods in the frequency domain involve considerable human effort. In this paper, we present a new method to automatically design CFAs in the frequency domain. Our method is based on the frequency structure representation of mosaicked images. We utilize a multi-objective optimization approach to propose frequency structure candidates, in which the overlap among the frequency components of images mosaicked with the CFA is minimized. Then, we optimize parameters for each candidate, which is formulated as a constrained optimization problem. We use the alternating direction method to solve it. Our parameter optimization method is applicable to arbitrary frequency structures, including those with conjugate replicas of chrominance components. Experiments on benchmark images confirm the advantage of the proposed method.

Demosaicking is the problem of reconstructing a color image from the raw image captured by a digital color camera that covers its only imaging sensor with a color filter array (CFA). Sparse representation-based demosaicking has been shown to produce superior reconstruction quality. However, almost all existing algorithms in this category use the CFAs, which are not specifically optimized for the algorithms. In this paper, we consider optimally designing CFAs for sparse representation-based demosaicking, where the dictionary is well-chosen. The fact that CFAs correspond to the projection matrices used in compressed sensing inspires us to optimize CFAs via minimizing the mutual coherence. This is more challenging than that for traditional projection matrices because CFAs have physical realizability constraints. However, most of the existing methods for minimizing the mutual coherence require that the projection matrices should be unconstrained, making them inapplicable for designing CFAs. We consider directly minimizing the mutual coherence with the CFA's physical realizability constraints as a generalized fractional programming problem, which needs to find sufficiently accurate solutions to a sequence of nonconvex nonsmooth minimization problems. We adapt the redistributed proximal bundle method to address this issue. Experiments on benchmark images testify to the superiority of the proposed method. In particular, we show that a simple sparse representation-based demosaicking algorithm with our specifically optimized CFA can outperform LSSC [1]. To the best of our knowledge, it is the first sparse representation-based demosaicking algorithm that beats LSSC in terms of CPSNR.

The Penrose pixel layout, an aperiodic pixel layout in rhombus Penrose tiling, has been shown to substantially outperform the existing square pixel layout in super-resolution. However, it was tested only on grayscale images. To study its performance on color images, we have to reconstruct regular color images from Penrose raw images, i.e., images with only one color component at each Penrose pixel, resulting in the problem of demosaicking from Penrose pixels. Penrose demosaicking is more difficult than regular demosaicking, because none of the color components of the reconstructed regular color images are available. Therefore, most of the traditional demosaicking methods do not apply. We develop a sparse representation-based method for Penrose demosaicking. Extensive experiments show that Penrose pixel layout outperforms regular pixel layouts in terms of both perceptual evaluation and S-CIELAB. The Penrose pixel layout is unique among all irregular layouts because it is uniformly three-colorable and it has only two pixel shapes, thick and thin rhombi, making its manufacturing relatively easy.

In most of existing digital cameras, color images have to be reconstructed from raw images which only have one color sensed at each pixel, as their imaging sensors are covered by color filter arrays (CFAs). At each pixel a CFA usually allows only a portion of the light spectrum to pass through and thereby reduces the light sensitivity of pixels. To address this issue, previous works have explored adding panchromatic pixels into CFAs. However, almost all existing methods assign panchromatic pixels empirically, making the designed CFAs prone to aliasing artifacts. In this paper, based on a mathematical model we propose a fully automatic approach to designing high-sensitivity CFAs using panchromatic pixels. By the frequency structure representation of CFAs, we formulate high-sensitivity CFA design as a continuous multi-objective optimization problem, where robustness to aliasing artifacts and percentage of panchromatic pixels are simultaneously maximized. We analyze the characteristics of our new formulation. According to the analysis, we develop a new method to propose frequency structure candidates, which can produce CFAs that reach a desired percentage of panchromatic pixels. Then for each candidate, we optimize parameters to obtain the final CFA, which is an appropriately balanced solution to the multi-objective optimization problem. We formulate the two design procedures as constrained optimization problems and solve them using the alternating direction method (ADM). Extensive experiments confirm the advantage of the proposed method in both low-light and normal-light conditions.

No abstract