Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network

Abstract

Large-scale linear operations are the cornerstone for performing complex computational tasks. Using optical computing to perform linear transformations offers potential advantages in terms of speed, parallelism, and scalability. Previously, the design of successive spatially engineered diffractive surfaces forming an optical network was demonstrated to perform statistical inference and compute an arbitrary complex-valued linear transformation using narrowband illumination. We report deep-learning-based design of a massively parallel broadband diffractive neural network for all-optically performing a large group of arbitrarily selected, complex-valued linear transformations between an input and output field of view, each with N i and N o pixels, respectively. This broadband diffractive processor is composed of N w wavelength channels, each of which is uniquely assigned to a distinct target transformation; a large set of arbitrarily selected linear transformations can be individually performed through the same diffractive network at different illumination wavelengths, either simultaneously or sequentially (wavelength scanning). We demonstrate that such a broadband diffractive network, regardless of its material dispersion, can successfully approximate N w unique complex-valued linear transforms with a negligible error when the number of diffractive neurons (N) in its design is ≥2N w N i N o . We further report that the spectral multiplexing capability can be increased by increasing N; our numerical analyses confirm these conclusions for N w > 180 and indicate that it can further increase to N w ∼ 2000, depending on the upper bound of the approximation error. Massively parallel, wavelength-multiplexed diffractive networks will be useful for designing highthroughput intelligent machine-vision systems and hyperspectral processors that can perform statistical inference and analyze objects/scenes with unique spectral properties.

Data‐Class‐Specific All‐Optical Transformations and Encryption

“…Furthermore, some of the emerging digital imaging techniques that are optimized to see through random scattering media are unlikely to compromise our data‐class encryption scheme enabled by diffractive networks due to their requirement of prior knowledge of the scattering medium properties [ 38–41 ] or the input‐output measurement pairs. [ 42–45 ] In addition to these, due to the absorbing/blocking areas of a D 2 NN, its architecture does not present time‐reversal symmetry; [ 21 ] when this feature is combined with the fact that the optical phase information is lost at the image sensor‐array, it becomes unattainable for adversaries to decipher the original input objects through backward field propagation operations applied on the acquired intensity‐only images, further reinforcing our system's security.…”

“…Diffractive feature size and layer-to-layer separation were set such that this diffractive processor works with a numerical aperture (NA) of ≈1 and has adequate degrees of freedom to approximate the designed transformation tasks with high fidelity. [15,30,37] The number of diffractive layers used in our designs was empirically determined based on the task complexity, practical/experimental challenges, and diminishing performance improvement associated with additional layers. Numerical simulations showed that the performance of a D 2 NN improves with an increased number of diffractive layers due to the additional degrees of freedom that grant higher design flexibility and better approximation accuracy, by avoiding the power dominance of ballistic photons that diffract within a low NA.…”

“…The latter, D 2 NNs or diffractive optical networks, [ 4,6 ] form an emerging platform that uses deep learning methods to design diffractive surfaces for all‐optical or optoelectronic (hybrid) computing. Once they are fabricated (after their optimization), diffractive networks can perform complex computer vision tasks, such as image classification, [ 4,7–9 ] hologram reconstruction, [ 10 ] quantitative phase imaging (QPI), [ 11 ] imaging through diffusers, [ 12 ] and others [ 13–22 ] using light diffraction through passive structured layers, one layer following another within a compact volume. Especially for visual computing tasks, diffractive networks provide unique advantages compared to other photonic, electronic, or hybrid (optoelectronic) computing platforms; [ 23,24 ] diffractive networks do not need to preprocess visual information since they have direct access to and can process the 2D and 3D spatial information of a scene, encoded by e.g., amplitude, phase, polarization, and spectrum of the input light.…”

Unidirectional imaging using deep learning–designed materials

“…To further enhance the unidirectional imaging performance of these diffractive designs, one strategy would be to create deeper architectures with more diffractive layers, also increasing the total number ( N ) of trainable features. In general, deeper diffractive architectures present advantages in terms of their learning speed, output power efficiency, transformation accuracy, and spectral multiplexing capability ( 39 , 44 , 47 , 48 ). Suppose an increase in the space-bandwidth product (SBP) of the input FOV A (SBP A ) and the output FOV B (SBP B ) of the unidirectional imager is desired, for example, due to a larger input FOV and/or an improved resolution demand; in that case, this will necessitate an increase in N proportional to SBP A × SBP B , demanding larger degrees of freedom in the diffractive unidirectional imager to maintain the asymmetric optical mode processing over a larger number of input and output pixels.…”

Snapshot multispectral imaging using a diffractive optical network

“…Here, we introduce the design of a snapshot multispectral imager that is based on a diffractive optical network (also known as D 2 NN, diffractive deep neural network [51][52][53][54][55][56][57][58][59][60] ) and demonstrate its performance with 4 (2 × 2), 9 (3 × 3) and 16 (4 × 4) unique spectral bands that are periodically repeating at the output image FOV to form a virtual multispectral filter array. This diffractive network-based multispectral imager (Fig.…”