Fourier optical preprocessing in lieu of deep learning

Abstract

Deep learning convolutional neural networks generally involve multiple-layer, forward-backward propagation machine-learning algorithms that are computationally costly. In this work, we demonstrate an alternative scheme to convolutional neural nets that reconstructs an original image from its optically preprocessed, Fourier-encoded pattern. The scheme is much less computationally demanding and more noise robust, and thus suited for high-speed and low-light imaging. We introduce a vortex phase transform with a lenslet-array to accompany shallow, dense, “small-brain” neural networks. Our single-shot coded-aperture approach exploits the coherent diffraction, compact representation, and edge enhancement of Fourier-transformed spiral phase gradients. With vortex encoding, a small brain is trained to deconvolve images at rates 5–20 times faster than those achieved with random encoding schemes, where greater advantages are gained in the presence of noise. Once trained, the small brain reconstructs an object from intensity-only data, solving an inverse mapping without performing iterations on each image and without deep learning schemes. With vortex Fourier encoding, we reconstruct MNIST Fashion objects illuminated with low-light flux ( 5 nJ / cm 2 ) at a rate of several thousand frames per second on a 15 W central processing unit. We demonstrate that Fourier optical preprocessing with vortex encoders achieves similar accuracies and speeds 2 orders of magnitude faster than convolutional neural networks.

Toward simple, generalizable neural networks with universal training for low-SWaP hybrid vision

“…We have previously shown that a convolutional neural network can outperform a single layer neural network but with significantly higher energy cost. The deep neural network is also less robust to noise [29]. Here, we aim to work with a "small brain" neural network rather than a deep neural network architecture.…”

“…In this section, we review an approach similar to [29] for our study of generalizable training. Figure 1(b) shows a schematic of the hybrid machine vision system, which encodes the image prior to the neural network with either a random or vortex phase pattern.…”

“…Whereas biomedical applications are aimed at large-image, high-quality image reconstruction [3], we turn our attention towards building real-time computational cameras for low size, weight, and power (SWaP) image reconstruction, which are needed for autonomous-vehicle applications. In our prior effort [29], we demonstrate reconstruction with a "small brain" dual-layer neural network. Such regression-based approaches [30] demonstrate fast reconstruction rates, robustness to noise, and show potential for generalization with a phase vortex encoder.…”

Insect-inspired nanofibrous polyaniline multi-scale films for hybrid polarimetric imaging with scattered light

“…[20][21][22][23][24][25][26] Using both optics and computation reduces time, power, memory, and complexity requirements; this hybrid approach may also provide additional information via a single sensor shot, which is useful for image enhancement. [27][28][29][30] In this communication, we show that multi-scaled structures with both material anisotropy and surface patterning may be employed for polarization-sensitive computer vision applications. Our insect-inspired design stems from knowledge of the corneal nanostructures on compound eyes, [2][3][4] which would optically shape and sort light via scattering.…”

Intelligent designs in nanophotonics: from optimization towards inverse creation

“…This optical learning architecture achieved a high accuracy on the specific tasks (e.g., object classification and matrix-vector multiplication) close to in silico training on an electronic computer. Furthermore, there exist several studies on Fourier space image processing by optical neural network [124,131]. For instance, Yan et al [124] set up a Fourier space D 2 NN by placing an optical nonlinear activation function (introduced by ferroelectric thin films) in an 2f system (see Fig.…”