Spectrally encoded single-pixel machine vision using diffractive networks

Abstract

We demonstrate optical networks composed of diffractive layers trained using deep learning to encode the spatial information of objects into the power spectrum of the diffracted light, which are used to classify objects with a single-pixel spectroscopic detector. Using a plasmonic nanoantenna-based detector, we experimentally validated this single-pixel machine vision framework at terahertz spectrum to optically classify the images of handwritten digits by detecting the spectral power of the diffracted light at ten distinct wavelengths, each representing one class/digit. We also coupled this diffractive network-based spectral encoding with a shallow electronic neural network, which was trained to rapidly reconstruct the images of handwritten digits based on solely the spectral power detected at these ten distinct wavelengths, demonstrating task-specific image decompression. This single-pixel machine vision framework can also be extended to other spectral-domain measurement systems to enable new 3D imaging and sensing modalities integrated with diffractive network-based spectral encoding of information.

Computational imaging without a computer: seeing through random diffusers at the speed of light

“…Hybrid systems that utilize diffractive networks as a front-end of a jointly-trained electronic neural network (back-end) [74] is another exciting future research direction that will make use of the presented framework to see through more complicated, dynamic scatters. Application of the presented framework and the underlying methodology to design broadband diffractive networks [66,67,76] is another exciting future research direction that can be used to reconstruct multi-color images distorted by unknown, random diffusers or other aberration sources. Finally, our results and presented method can be extended to other parts of the electromagnetic spectrum including e.g., visible/infrared wavelengths, and will open up various new applications in biomedical imaging, astronomy, astrophysics, atmospheric sciences, security, robotics, and many others.…”

“…Some of the more recent work on imaging through diffusers has also focused on using deep learning methods to digitally recover the images of unknown objects [11,12,48,49]. Deep learning has been re-defining the state-of-the-art across many areas in optics, including optical microscopy [50][51][52][53][54][55], holography [56][57][58][59][60][61], inverse design of optical devices [62][63][64][65][66][67], optical computation and statistical inference [68][69][70][71][72][73][74][75][76][77], among others [78][79][80].…”

Roadmap on Digital Holography-Based Quantitative Phase Imaging

“…Ultimately, optical setup and computers can be discarded. Ozcan et al introduced an all-optical diffractive NN able to compute a classification task from spatial information of objects [176]. In line with this development, novel all-optical diffractive NNs could perform quantitative measurements (Figure 12c).…”

Ti3C2Tx MXene Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

“…In particular, where any unitary transformations can be implemented with conventional optical beamsplitters and phase shifters 35 , a rectangular diagonal matrix can be implemented with optical attenuation achieved by Mach-Zehnder modulator. As well, an OIU implementation that relies on free-space diffractive DNN [36][37][38] has already been shown in the spectral domain 39,40 ). Implementations of optical nonlinearity unit (ONLU) fall into two major categories, one based on optical-to-electrical-to-optical (OEO) and the other on alloptical (AO).…”