Unidirectional imaging using deep learning–designed materials

Li, Jingxi; Gan, Tianyi; Zhao, Yifan; Bai, Bijie; Shen, Che‐Yung; Sun, Songyu; Jarrahi, Mona; Ozcan, Aydogan

doi:10.1126/sciadv.adg1505

Cited by 30 publications

(8 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning-optimized diffractive processors show promise in direct suppression of speckle noise under coherent illumination conditions 51 . In principle, diffractive processors can also be designed to function under partially coherent or incoherent illumination to mitigate different forms of noise with different statistical features 53 , 54 .…”

Section: Discussionmentioning

confidence: 99%

All-optical image denoising using a diffractive visual processor

Işıl,

Gan,

Ardic

et al. 2024

Light Sci Appl

Self Cite

View full text Add to dashboard Cite

Image denoising, one of the essential inverse problems, targets to remove noise/artifacts from input images. In general, digital image denoising algorithms, executed on computers, present latency due to several iterations implemented in, e.g., graphics processing units (GPUs). While deep learning-enabled methods can operate non-iteratively, they also introduce latency and impose a significant computational burden, leading to increased power consumption. Here, we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images – implemented at the speed of light propagation within a thin diffractive visual processor that axially spans <250 × λ, where λ is the wavelength of light. This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features, causing them to miss the output image Field-of-View (FoV) while retaining the object features of interest. Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of ~30–40%. We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum. Owing to their speed, power-efficiency, and minimal computational overhead, all-optical diffractive denoisers can be transformative for various image display and projection systems, including, e.g., holographic displays.

show abstract

Section: Discussionmentioning

confidence: 99%

All-optical image denoising using a diffractive visual processor

Işıl,

Gan,

Ardic

et al. 2024

Light Sci Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our previous research, we developed diffractive processor designs tailored for imaging either amplitude distributions of amplitude-only objects 51 or phase distributions of phase-only objects 53 , 59 , 60 . However, these designs would become ineffective for imaging complex objects with independent and non-uniform distributions in the amplitude and phase channels.…”

Section: Discussionmentioning

confidence: 99%

All-optical complex field imaging using diffractive processors

Li,

Gan

et al. 2024

Light Sci Appl

Self Cite

View full text Add to dashboard Cite

Complex field imaging, which captures both the amplitude and phase information of input optical fields or objects, can offer rich structural insights into samples, such as their absorption and refractive index distributions. However, conventional image sensors are intensity-based and inherently lack the capability to directly measure the phase distribution of a field. This limitation can be overcome using interferometric or holographic methods, often supplemented by iterative phase retrieval algorithms, leading to a considerable increase in hardware complexity and computational demand. Here, we present a complex field imager design that enables snapshot imaging of both the amplitude and quantitative phase information of input fields using an intensity-based sensor array without any digital processing. Our design utilizes successive deep learning-optimized diffractive surfaces that are structured to collectively modulate the input complex field, forming two independent imaging channels that perform amplitude-to-amplitude and phase-to-intensity transformations between the input and output planes within a compact optical design, axially spanning ~100 wavelengths. The intensity distributions of the output fields at these two channels on the sensor plane directly correspond to the amplitude and quantitative phase profiles of the input complex field, eliminating the need for any digital image reconstruction algorithms. We experimentally validated the efficacy of our complex field diffractive imager designs through 3D-printed prototypes operating at the terahertz spectrum, with the output amplitude and phase channel images closely aligning with our numerical simulations. We envision that this complex field imager will have various applications in security, biomedical imaging, sensing and material science, among others.

show abstract

“…As shown in Figure 4, diffractive deep neural networks (D 2 NNs) have emerged as a free-space optical platform that leverages supervised deep learning algorithms to design diffractive surfaces for visual processing and all-optical computational tasks [70,71]. These diffractive optical networks, after their fabrication, form physical all-optical processors of visual information capable of executing various computer vision tasks spanning image classification [70,[72][73][74][75], quantitative phase imaging (QPI) [76,77], linear transformations [78][79][80][81], image encryption [82,83], and imaging through diffusive media [84,85], among many others [86][87][88][89][90][91][92][93]. Each point on the diffractive layer represents an artificial neuron in which the phase or amplitude can be independently modulated, and the field distribution is…”

Section: Diffractive and Optoelectronic Neural Networkmentioning

confidence: 99%

Advances in Mask-Modulated Lensless Imaging

Wang,

Duan

2024

Electronics

View full text Add to dashboard Cite

Lensless imaging allows for designing imaging systems that are free from the constraints of traditional imaging architectures. As a broadly investigated technique, mask-modulated lensless imaging encodes light signals via a mask plate integrated with the image sensor, which is more compacted, with scalability and compressive imaging abilities. Here, we review the latest advancements in mask-modulated lensless imaging, lensless image reconstruction algorithms, related techniques, and future directions and applications.

show abstract

Unidirectional imaging using deep learning–designed materials

Cited by 30 publications

References 53 publications

All-optical image denoising using a diffractive visual processor

All-optical image denoising using a diffractive visual processor

All-optical complex field imaging using diffractive processors

Advances in Mask-Modulated Lensless Imaging

Contact Info

Product

Resources

About