On a mathematical theory of coded exposure

Abstract

This paper proposes a mathematical model and formalism to study coded exposure (flutter shutter) cameras. The model includes the Poisson photon (shot) noise as well as any additive (readout) noise of finite variance. This is an improvement compared to our previous work that only considered the Poisson noise. Closed formulae for the mean square error and signal to noise ratio of the coded exposure method are given. These formulae take into account for the whole imaging chain, i.e., the Poisson photon (shot) noise, any additive (readout) noise of finite variance as well as the deconvolution and are valid for any exposure code. Our formalism allows us to provide a curve that gives an absolute upper bound for the gain of any coded exposure camera in function of the temporal sampling of the code. The gain is to be understood in terms of mean square error (or equivalently in terms of signal to noise ratio), with respect to a snapshot (a standard camera). Keywords: Coded exposure, Computational photography, Flutter shutter, Motion blur, Mean square error (MSE), Signal to noise ratio (SNR) Mathematics Subject Classification: Primary 68U10; Secondary 42A38, 60G99 BackgroundSince the seminal papers [1][2][3][4][5][6] of Agrawal and Raskar coded exposure (flutter shutter) method has received a lot of follow-ups . In a nutshell, the authors proposed to open and close the camera shutter, according to a sequence called "code," during the exposure time. By this clever exposure technique, the coded exposure method permits one to arbitrarily increase the exposure time when photographing (flat) scenes moving at a constant velocity. Note that with a coded exposure method only one picture is stored/transmitted. A rich body of empirical results suggest that the coded exposure method allows for a gain in terms of Mean Square Error (MSE) or Signal to Noise Ratio (SNR) compared to a classic camera, i.e., a snapshot. Therefore, the coded exposure method seems to be a magic tool that should equip all cameras.We now briefly expose the different applications, variants, and studies that surround the coded exposure method. An application of the coded exposure method to bar codes is given in [16,35], to fluorescent cell image in [27], to periodic events in [25,34,36], to multispectral imaging in [10], and to iris in [21]. Application to motion estimation/deblurring are presented in [9,19,20,26,31,33,37,38]. An extension for space-dependent blur is investigated in [28]. Methods to find better or optimal sequences as investigated in [12-© 2016 Tendero and Osher. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. 0123456789().,-: vol Tendero and Osher Res Math Sci (2016) 3:4Page 2 of 39 14,22,23,39] o...

On Hamilton-Jacobi PDEs and image denoising models with certain non-additive noise

“…Then, (42) follows from (49) and ( 52). Now, we calculate the interior of dom S. Since dom S is a subset of R n × [0, +∞), there holds…”

“…To be specific, let v be the gray level array of the original image, and assume there is no motion and the gray level image v does not change over time. The observed image is a sample from a Poisson distribution whose rate equals tv, where t is the exposure time of the sensor (see [42]). The probability mass function of the Poisson distribution at x ∈ Z n equals…”

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

“…In contrast, TEGI consists of many short flashes with carefully chosen intensity and time points, which largely enhances the overall light throughout. Since the advent of temporally coded exposure, numerous works have sought to better understand the underlying mathematical theory [67], to improve the technique's speed and accuracy [68], and to extend its application to spatially varying blurring [69]. With faster and brighter stroboscopic light sources, the motion deblurring ability could be further extended to fast and arbitrarily moving objects [62].…”

Motion Deblurring using Spatiotemporal Phase Aperture Coding

“…Moreover, since it also relies on temporal amplitude coding, light efficiency is still decreased. A rigorous model analyzing the design and implementation of a fluttered shutter camera is presented in [43,44]. Jeon et al extended the method to multi-image photography using complementary fluttering patterns [17].…”