Toward simple, generalizable neural networks with universal training for low-SWaP hybrid vision

Abstract

Speed, generalizability, and robustness are fundamental issues for building lightweight computational cameras. Here we demonstrate generalizable image reconstruction with the simplest of hybrid machine vision systems: linear optical preprocessors combined with nohidden-layer, "small-brain" neural networks. Surprisingly, such simple neural networks are capable of learning the image reconstruction from a range of coded diffraction patterns using two masks. We investigate the possibility of generalized or "universal training" with these small brains. Neural networks trained with sinusoidal or random patterns uniformly distribute errors around a reconstructed image, whereas models trained with a combination of sharp and curved shapes (the phase pattern of optical vortices) reconstruct edges more boldly. We illustrate variable convergence of these simple neural networks and relate learnability of an image to its singular value decomposition entropy (SVD-entropy) of the image. We also provide heuristic experimental results. With thresholding, we achieve robust reconstruction of various disjoint datasets. Our work is favorable for future real-time low-SWaP hybrid vision: we reconstruct images on a 15W laptop CPU with 15k fps: faster by a factor of 3 than previously reported results and 3 orders of magnitude faster than convolutional neural networks.

Feature and texture distillation via neural network training

“…We train image datasets with different fractal and random training datasets including speckle patterns. We show that the resulting model distils and achieves image segmentation not only for objects (which we have achieved previously [7]) but also for textures. Finally, when we reclassify these images, we achieve order-ofmagnitude improvements in the accuracies, many of which rival decade-old computer vision state-of-the-art.…”

Deep learning in photonics: introduction

“…Representative examples include deep compressive imaging techniques via optimized-pattern scanning [19], compressed ultrafast photography via an augmented-Lagrangian and deep-learning hybrid algorithm [20], and deep plug-and-play priors for spectral snapshot compressive imaging [21]. Image reconstruction techniques include denoising and reconstruction of super-resolution structured illumination microscopy images [22], and a simple low-SWaP hybrid machine vision system for universal training and generalized image reconstruction [23]. Novel imaging strategies through unknown scattering media based on physics informed learning [24], and incoherent imaging through highly dynamic and optically thick turbid media [25] are also demonstrated.…”