A Bit Too Much? High Speed Imaging from Sparse Photon Counts

Abstract

Recent advances in photographic sensing technologies have made it possible to achieve light detection in terms of a single photon. Photon counting sensors are being increasingly used in many diverse applications. We address the problem of jointly recovering spatial and temporal scene radiance from very few photon counts. Our ConvNet-based scheme effectively combines spatial and temporal information present in measurements to reduce noise. We demonstrate that using our method one can acquire videos at a high frame rate and still achieve good quality signal-to-noise ratio. Experiments show that the proposed scheme performs quite well in different challenging scenarios while the existing approaches are unable to handle them.

Image Classification in the Dark using Quanta Image Sensors

“…Readers interested in the sensor development can consult recent keynote reports, e.g., [22]. On the algorithmic side, a number of theoretical signal processing results and reconstruction algorithms have been proposed [3], [9], [23], including some very recent methods based on deep learning [7], [11]. However, since the sensor is relatively new, computer vision applications of the sensor are not yet common.…”

“…By oversampling the space and time, and by using a carefully designed image reconstruction algorithm, QIS can capture very low-light images with signal-to-noise ratio much higher than existing CMOS image sensors of the same pixel pitch [3]. Over the past few years, prototype QIS have been built by researchers at Dartmouth and Gigajot Technology Inc. [4], [5], with a number of theoretical and algorithmic contributions by researchers at EPFL [6], [7], Harvard [8], and Purdue [9]- [13]. Today, the latest QIS prototype can perform color imaging with a read noise of 0.25e − /pix (compared to at least several electrons in CIS [14]) and dark current of 0.068e − /pix/s at room temperature (compared to > 1e − /pix/s in CIS) [5], [9].…”

Quanta Burst Photography

“…Image reconstruction from single-photon sensor data. There is prior work on reconstructing intensity images from single-photon binary frames using denoising techniques such as total variation and BM3D [Chan et al 2016;, or by an end-toend neural network [Chandramouli et al 2019;Choi et al 2018].…”

Megapixel photon-counting color imaging using quanta image sensor

“…In the gray-scale setting, we can formulate the problem as maximum-likelihood and solve it using convex optimization tools [9,11,12,25]. We can also use learning-based methods, e.g., [26][27][28][29] to reconstruct the signal. The method we present here is based on the transform-denoise approach by Chan et al [10].…”