Image Systems Simulation

Farrell, Joyce E.; Wandell, Brian A.

doi:10.1002/9781118798706.hdi012

Cited by 16 publications

(10 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ISETCam [Ref. 8] software converts the spectral irradiance data into pixel voltages and digital outputs (Farrell, Catrysse, and Wandell 2012;Farrell and Wandell 2015). Using ISETCam we can simulate different types of sensors, varying parameters such as the color filter array and pixel size (Figure 7).…”

Section: Sensor Modelingmentioning

confidence: 99%

A system for generating complex physically accurate sensor images for automotive applications

Liu¹,

Shen²,

Zhang³

et al. 2019

View full text Add to dashboard Cite

We describe an open-source simulator that creates sensor irradiance and sensor images of typical automotive scenes in urban settings. The purpose of the system is to support camera design and testing for automotive applications. The user can specify scene parameters (e.g., scene type, road type, traffic density, time of day) to assemble a large number of random scenes from graphics assets stored in a database. The sensor irradiance is generated using quantitative computer graphics methods, and the sensor images are created using image systems sensor simulation. The synthetic sensor images have pixel level annotations; hence, they can be used to train and evaluate neural networks for imaging tasks, such as object detection and classification. The end-to-end simulation system supports quantitative assessment -from scene to camera to network accuracy -for automotive applications.

show abstract

Section: Sensor Modelingmentioning

confidence: 99%

A system for generating complex physically accurate sensor images for automotive applications

Liu¹,

Shen²,

Zhang³

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The ISETCam software converts the sensor spectral irradiance to RGB images ( Figure 2) [13,11,12]. The sensor parameters for the simulations reported in this paper were based on the MT9V024 sensor manufactured by ON Semiconductor.…”

Section: Camera Simulationmentioning

confidence: 99%

Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network

Liu

Lian

Farrell

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Figure 1. Simulated scenes and metadata. An ISE3d generated scene with sky maps measured in (A) early afternoon and (B) late afternoon. The simulation creates pixel-level labeled images including (C) a panoptic segmentation, and (D) a depth map. (E) A simulated scene with superimposed graphs showing the spectral radiance at different image locations (circles). The spectral irradiance is used by the camera simulation to account for the wavelength effects of the lens and color filters. AbstractImaging systems are increasingly used as input to convolutional neural networks (CNN) for object detection; we would like to design cameras that are optimized for this purpose. It is impractical to build different cameras and then acquire and label the necessary data for every potential camera design; creating software simulations of the camera in context (soft prototyping) is the only realistic approach. We implemented soft-prototyping tools that can quantitatively simulate image radiance and camera designs to create realistic images that are input to a convolutional neural network for car detection. We used these methods to quantify the effect that critical hardware components (pixel size), sensor control (exposure algorithms) and image processing (gamma and demosaicing algorithms) have upon average precision of car detection. We quantify (a) the relationship between pixel size and the ability to detect cars at different distances, (b) the penalty for choosing a poor exposure duration, and (c) the ability of the CNN to perform car detection for a variety of post-acquisition processing algorithms. These results show that the optimal choices for car detection are not constrained by the same metrics used for image quality in consumer photography. It is better to evaluate camera designs for CNN applications using soft prototyping with task-specific metrics rather than consumer photography metrics.

show abstract

“…For classical color imaging simulation, reflectance data represented on MCC chart may be used to compute a CCM or adjust white balance (WB) of the imaging device [5]. This chart consists of 24 color patches that represent some physical common colors such as skin or foliage colors… sRGB and L*a*b* values of each patch are given by the manufacturer under respectively D65 and D50 CIE illuminants.…”

Section: Standard Color Charts and Limitation For Vis-nir Color Restimentioning

confidence: 99%

“…For each monochromatic image, dark current and flat field correction are applied. Knowing all physical acquisition parameters such as optical setup, spectral sensitivity of the acquisition sensor and the illuminance spectrum, the reflectance spectra are extracted [5].…”

Section: Vis-nir Reflectance Spectra Collectionmentioning

confidence: 99%

Color correction matrix for sparse RGB-W image sensor without IR cutoff filter

Vaillant

Clouet

Alleysson

2018

Unconventional Optical Imaging

View full text Add to dashboard Cite

Unlike photographic image sensors with infrared cutoff filter, low light image sensors gather light over visible and near infrared (VIS-NIR) spectrum to improve sensitivity. However, removing infrared cutoff filter makes the color rendering challenging. In addition, no color chart, with calibrated infrared content, is available to compute color correction matrix (CCM) of such sensors. In this paper we propose a method to build a synthetic color chart (SCC) to overcome this limitation. The choice of chart patches is based on a smart selection of spectra from open access and our own VIS-NIR hyperspectral images databases. For that purpose we introduce a fourth cir dimension to CIE-L*a*b* space to quantify the infrared content of each spectrum. Then we uniformly sample this L*a*b*cir space, leading to 1498 spectra constituting our synthetic color chart. This new chart is used to derive a 3x4 color correction matrix associated to the commercial RGB-White sensor (Teledyne-E2V EV76C664) using a classical linear least square minimization.. We show an improvement of signal to noise ratio (SNR) and color accuracy at low light level compared to standard CCM derived using Macbeth color chart.

show abstract

Image Systems Simulation

Abstract: We describe a suite of software tools for image system simulation. The tools model the three-dimensional scene radiance, image formation by the optics, sensor transduction, image processing and display rendering.

Cited by 16 publications

References 79 publications

A system for generating complex physically accurate sensor images for automotive applications

A system for generating complex physically accurate sensor images for automotive applications

Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network

Color correction matrix for sparse RGB-W image sensor without IR cutoff filter

Contact Info

Product

Resources

About