Efficient image reconstruction for gigapixel quantum image sensors

Abstract

Recent advances in materials, devices and fabrication technologies have motivated a strong momentum in developing solid-state sensors that can detect individual photons in space and time. It has been envisioned that such sensors can eventually achieve very high spatial resolutions (e.g., 109 pixels/chip) as well as high frame rates (e.g., 106 frames/sec). In this paper, we present an efficient algorithm to reconstruct images from the massive binary bit-streams generated by these sensors. Based on the concept of alternating direction method of multipliers (ADMM), we transform the computationally intensive optimization problem into a sequence of subproblems, each of which has efficient implementations in the form of polyphase-domain filtering or pixel-wise nonlinear mappings. Moreover, we reformulate the original maximum likelihood estimation as maximum a posterior estimation by introducing a total variation prior. Numerical results demonstrate the strong performance of the proposed method, which achieves several dB's of improvement in PSNR and requires a shorter runtime as compared to standard gradient-based approaches.

Images from Bits: Non-Iterative Image Reconstruction for Quanta Image Sensors

“…We generate one-bit observations using a quantized Poisson statistics and use the proposed algorithm to reconstruct. As a comparison, we also test the MLE solution, i.e., a summation followed by the inverse incomplete Gamma transform (see Theorem 1), and an ADMM algorithm using a total variation regularization [29,31]. For the proposed algorithm, we use BM3D as the image denoiser.…”

“…A similar description of the model was previously discussed in [13,29]. For notational simplicity, we consider one-dimensional signals.…”

“…However, for general $\mathrm{bold-italicG}$ or if we include additional constraints to enforce the smoothness of the solution, e.g., using total variation [34] or sparsity [35], then numerical algorithms are required. Some existing methods include the gradient descent method [13], Newton method [28] and the alternating direction method of multipliers [29]. These algorithms are iterative, and the computation for each iteration is costly.…”

“…Image reconstruction is a critical component of QIS, for without such an algorithm, we will not be able to form images. However, unlike classical Poisson image recovery problems where solutions are abundant [22,23,24,25,26,27], the one-bit quantization of QIS makes the problem uniquely challenging, and there is a limited number of existing methods [28,29,30,31]. Another challenge we have to overcome is the complexity of the algorithm, which has to be low enough that we can put them on cameras to minimize power consumption, memory consumption and runtime.…”

Megapixel photon-counting color imaging using quanta image sensor

“…The task of image reconstruction is to recover color scene c from the measurements B. In the gray-scale setting, we can formulate the problem as maximum-likelihood and solve it using convex optimization tools [9,11,12,25]. We can also use learning-based methods, e.g., [26][27][28][29] to reconstruct the signal.…”