DeepGhost: real-time computational ghost imaging via deep learning

Abstract

the potential of random pattern based computational ghost imaging (cGi) for real-time applications has been offset by its long image reconstruction time and inefficient reconstruction of complex diverse scenes. to overcome these problems, we propose a fast image reconstruction framework for cGi, called "DeepGhost", using deep convolutional autoencoder network to achieve real-time imaging at very low sampling rates (10-20%). By transferring prior-knowledge from STL-10 dataset to physical-data driven network, the proposed framework can reconstruct complex unseen targets with high accuracy. The experimental results show that the proposed method outperforms existing deep learning and state-of-the-art compressed sensing methods used for ghost imaging under similar conditions. the proposed method employs deep architecture with fast computation, and tackles the shortcomings of existing schemes i.e., inappropriate architecture, training on limited data under controlled settings, and employing shallow network for fast computation. Computational ghost imaging 1 acquires spatial information about an unknown target by illuminating it with a series of random binary patterns generated by a spatial light modulator (SLM). For each projected pattern, the light intensity back-reflected from the target plane is recorded by an ordinary photodiode. By correlating intensity measurements with corresponding projected patterns, the target image is reconstructed. One downside of CGI is the requirement of a large number of measurements to produce a good-quality image, which increases its imaging time. Despite the emergence of basis scan schemes 2 , CGI (using random patterns) is still employed in many applications due to its simplicity, inherent encryption of patterns 3 , and ease of deployment 4. Therefore, it is important to improve the efficiency of CGI by integrating it with some optimization technique to avoid complex (hardware based) methods 5 that fail to reap the benefits of reduced cost and simplicity in ghost imaging (GI). Owing to its advantages of low cost, robustness against noise and scattering, and ability to operate over long spectral range, CGI is widely used in many applications 6-8. In order to make CGI practical, more specifically for real-time imaging, it is important to reduce its imaging time. The imaging time of CGI can be subcategorized as data acquisition time and image reconstruction time. The data acquisition time of CGI depends on the required number of measurements and mainly on the projection rate of SLM. Recent advances in SLM technology make it easy to reduce data acquisition time by employing commercially available high-resolution digital micromirror devices (DMDs) operating at ~ 20 kHz. The acquisition time can also be reduced by employing some simple yet novel solutions 9,10. Therefore, the image reconstruction time remains the main bottleneck towards achieving high speed imaging in CGI. This image reconstruction time can be reduced by employing an efficient image reconstruction framework. Recently, comp...

Ghost Imaging of Moving Target Based on the Periodic Pseudo-Thermal Light Field Generated by a 2D Silicon OPA

“…Moreover, the main limitation of traditional imaging schemes is that they provide only a limited FOV and have restricted image reconstruction capabilities for targets moving over a large area [27,28]. Finally, one advantage of OPA-based GI of moving targets is that a combination of many mature algorithms is feasible [32][33][34][35]. An OPA enables beam scanning for velocity extraction of moving targets and can be combined with computational imaging.…”

Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling

“…We note that the proposed sampling strategy requires CS for image reconstruction and CS algorithms are commonly computationally exhausted. In our future work, we consider using deep learning [38][39][40][41][42][43][44][45] to reconstruct the final image from the undersampled Fourier spectrum so as to reduce the image reconstruction time.…”

Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy

“…Then, the use of deep learning for restoration of the original image from data reconstructed with a small number of illuminations has been largely explored in the last five years. Typical approaches are the construction of neural network to reduce the number of illuminations and improve the restoration accuracy (fully-connected neural network [18], U-net [19], convolutional neural network [20], recurrent neural network [21,22]), and acceleration of image acquisition which aims at real-time performance by reducing the number of illuminations [23][24][25]. Most of these previous studies have shown versatile restoration performance using digits or natural images on databases such as MNIST [26] or ImageNet [27].…”