Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy

Cheng, Yi Fei; Strachan, Megan; Weiss, Zachary; Deb, Moniher; Carone, Dawn M.; Ganapati, Vidya

doi:10.1364/oe.27.000644

Cited by 59 publications

(33 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, deep learning receives much attention in many research fields including optical design [1,2] and optical imaging [3]. In previous works, deep learning has been extensively applied for many optical imaging problems including phase retrieval [4][5][6][7], microscopic image enhancement [8][9], scattering imaging [10][11], holography [12][13][14][15][16][17][18], single-pixel imaging [19,20], super-resolution [21][22][23][24], Fourier ptychography [25][26][27], optical interferometry [28,29], wavefront sensing [30,31], and optical fiber communications [32].…”

Section: Introductionmentioning

confidence: 99%

Does deep learning always outperform simple linear regression in optical imaging?

et al. 2020

View full text Add to dashboard Cite

Deep learning has been extensively applied in many optical imaging problems in recent years. Despite the success, the limitations and drawbacks of deep learning in optical imaging have been seldom investigated. In this work, we show that conventional linear-regression-based methods can outperform the previously proposed deep learning approaches for two black-box optical imaging problems in some extent. Deep learning demonstrates its weakness especially when the number of training samples is small. The advantages and disadvantages of linear-regression-based methods and deep learning are analyzed and compared. Since many optical systems are essentially linear, a deep learning network containing many nonlinearity functions sometimes may not be the most suitable option.

show abstract

Section: Introductionmentioning

confidence: 99%

Does deep learning always outperform simple linear regression in optical imaging?

et al. 2020

View full text Add to dashboard Cite

show abstract

“…By illuminating multiple LEDs at a time for each captured image, it was shown that the incoherent superposition of the contribution of each LED could be separated within a novel reconstruction algorithm. After this demonstration, several strategies were proposed to choose alternative multiplexing LED patterns [128][129][130][131][132], and similar approaches were also later suggested for spectral multiplexing [79,133]. One of the fastest FP systems to date was demonstrated using this multiplexing concept [26].…”

Section: High-speed Fourier Ptychographymentioning

confidence: 99%

“…Some works have also demonstrated LED multiplexing or compressive sensing to reduce the number of raw measurements needed [130,181,182,184,185]. Finally, a few works have used deep learning to find the optimal illumination pattern for compressive reconstructions [131,186,187] or for application-dependent tasks [188,189]. A comparison among these different approaches is summarized in Fig.…”

Section: Deep Learning In Fourier Ptychographymentioning

confidence: 99%

Fourier ptychography: current applications and future promises

et al. 2020

View full text Add to dashboard Cite

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

show abstract

“…For microscopic imaging, similar concepts to our approach have been applied to co-optimize a spatial light modulator to improve multi-color localization of fluorescent emitters [28][29][30]. In addition, several works have also optimized the illumination in a microscope via machine learning to improve imaging performance, primarily for the task of producing a phase contrast or quantitative phase image [31][32][33], but also for improving resolution-enhancement methods like Fourier ptychography [34][35][36]. A useful review of similar machine learning methods in microscopy is presented in Ref.…”

Section: Deep Learning For Optimal Illuminationmentioning

confidence: 99%

Learned sensing: jointly optimized microscope hardware for accurate image classification

Muthumbi

Chaware

Kim

et al. 2019

Biomed. Opt. Express

View full text Add to dashboard Cite

Since its invention, the microscope has been optimized for interpretation by a human observer. With the recent development of deep learning algorithms for automated image analysis, there is now a clear need to redesign the microscope's hardware for specific interpretation tasks. To increase the speed and accuracy of automated image classification, this work presents a method to co-optimize how a sample is illuminated in a microscope, along with a pipeline to automatically classify the resulting image, using a deep neural network. By adding a "physical layer" to a deep classification network, we are able to jointly optimize for specific illumination patterns that highlight the most important sample features for the particular learning task at hand, which may not be obvious under standard illumination. We demonstrate how our learned sensing approach for illumination design can automatically identify malaria-infected cells with up to 5-10% greater accuracy than standard and alternative microscope lighting designs. We show that this joint hardware-software design procedure generalizes to offer accurate diagnoses for two different blood smear types, and experimentally show how our new procedure can translate across different experimental setups while maintaining high accuracy.

show abstract

Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy

Cited by 59 publications

References 28 publications

Does deep learning always outperform simple linear regression in optical imaging?

Does deep learning always outperform simple linear regression in optical imaging?

Fourier ptychography: current applications and future promises

Learned sensing: jointly optimized microscope hardware for accurate image classification

Contact Info

Product

Resources

About