Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum

Abstract

Abstract-We propose the concept of a generalized assorted pixel (GAP) camera, which enables the user to capture a single image of a scene and, after the fact, control the tradeoff between spatial resolution, dynamic range and spectral detail. The GAP camera uses a complex array (or mosaic) of color filters. A major problem with using such an array is that the captured image is severely under-sampled for at least some of the filter types. This leads to reconstructed images with strong aliasing. We make four contributions in this paper: 1) we present a comprehensive optimization method to arrive at the spatial and spectral layout of the color filter array of a GAP camera. 2) We develop a novel algorithm for reconstructing the under-sampled channels of the image while minimizing aliasing artifacts. 3) We demonstrate how the user can capture a single image and then control the tradeoff of spatial resolution to generate a variety of images, including monochrome, high dynamic range (HDR) monochrome, RGB, HDR RGB, and multispectral images. 4) Finally, the performance of our GAP camera has been verified using extensive simulations that use multispectral images of real world scenes. A large database of these multispectral images has been made available at http://www1.cs.columbia.edu/ CAVE/projects/gap_camera/ for use by the research community.Index Terms-Assorted pixels, color filter array, color reproduction, demosaicing, dynamic range, multispectral imaging, signal to noise ratio, skin detection, sub-micrometer pixels.

Edge detection in multispectral images using the n-dimensional self-organizing map

“…We discuss this observation in detail and provide a solution to the problem. We show that our variant performs better on images from the high-quality CAVE multispectral image database [7], which is of higher resolution than previously tested images.…”

“…This can result in edges introduced between pixels that are part of such a cluster. Figure 1 shows an example image from the CAVE database [7] and the rank image obtained by a 1-dimensional SOM. The SOM with n = 64 neurons was trained with 100 000 random samples from the image.…”

Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution

“…experiments with two different public databases. The first database [33], called the CAVE database, consists of 32 hyperspectral images of everyday objects. The 512 × 512 spectral images of the scenes are acquired at a wavelength interval of 10 nm in the range 400−700 nm.…”

“…2: RGB images from the databases. First row: Images from the CAVE database [33]. Second row: Images from the Harvard database [11].…”

A Robust Skin Colour Segmentation Using Bivariate Pearson Type IIαα (Bivariate Beta) Mixture Model

“…Another important difficulty in skin colour segmentation is that, the colour image senses work based on colour mosaic, which is assortment of different spectral filters [4]. Hence in skin colour segmentation the colour space plays a dominant role for feature extraction.…”