Towards an Intelligent Microscope: Adaptively Learned Illumination for Optimal Sample Classification

Chaware, Amey; Cooke, Colin; Kim, Kanghyun; Horstmeyer, Roarke

doi:10.1109/icassp40776.2020.9054477

Cited by 10 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…S12 of Supplement 1 for more details on optical sectioning SIM). By adding the freedom of arbitrary designable illumination across the FOV, our approach may also be used to design custom illumination and detection variables which can potentially enhance the efficiency of computational imaging [48].…”

Section: Discussionmentioning

confidence: 99%

Tunable SIM: observation at varying spatiotemporal resolutions across the FOV

Woo

Jung

Ahn

et al. 2020

Optica

View full text Add to dashboard Cite

To date, imaging systems have generally been designed to provide an even spatiotemporal resolution across the field of view (FOV). However, this becomes a fundamental limitation when we aim to simultaneously observe varying dynamics at different parts of the FOV. In conventional imaging systems, to capture fast dynamics occurring at only a small portion of the FOV, the entire imaging system's sampling rate must be increased. This is a major problem if different parts of the FOV must rather be imaged at high spatial resolutions beyond the diffraction limit and require a sacrifice in temporal resolution. To answer this unmet challenge, we propose tunable SIM, which enables adaptive modulation of spatiotemporally varying structured illumination across different parts of the FOV. Using tunable SIM, we exploit the varying and designable spatiotemporal resolution to demonstrate simultaneous measurements of subdiffraction-limited changes in the actin fine structure of U87MG-EGFP-CD9 cells and the fast viscous flow inducing these structural changes.

show abstract

Section: Discussionmentioning

confidence: 99%

Tunable SIM: observation at varying spatiotemporal resolutions across the FOV

Woo

Jung

Ahn

et al. 2020

Optica

View full text Add to dashboard Cite

show abstract

“…Rather than following the static optical setup of the microscope and postprocessing its acquired images, recent methods have focused on optimizing certain parts of the optical hardware itself. Several approaches focus on optimizing the illumination patterns of the microscope [5,13,17,27]. This research direction of jointly optimizing the forward optics (by learning illumination patterns) with the inverse reconstruction model has been able to reduce the data requirement in QPM [15,16].…”

Section: Related Workmentioning

confidence: 99%

“…Despite the significant performance boost in such methods, the limitations inherited from the hardware of the microscope set the upper performance bound [7]. Therefore, over the past decade, researchers have focused on joint deep-learning optimization of not only the reconstruction model but also the hardware of the microscope itself [3,5,13,17,27,36]. Nevertheless, all these methods' focus was to optimize a system that is already capable of a specific imaging task.…”

Section: Introductionmentioning

confidence: 99%

Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

Herath¹,

Haputhanthri²,

Hettiarachchi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Ever since the first microscope by Zacharias Janssen in the late 16th century, scientists have been inventing new types of microscopes for various tasks. Inventing a novel architecture demands years, if not decades, worth of scientific experience and creativity. In this work, we introduce Differentiable Microscopy (∂µ), a deep learning-based design paradigm, to aid scientists design new interpretable microscope architectures. Differentiable microscopy first models a common physics-based optical system however with trainable optical elements at key locations on the optical path. Using pre-acquired data, we then train the model end-to-end for a task of interest. The learnt design proposal can then be simplified by interpreting the learnt optical elements. As a first demonstration, based on the optical 4-f system, we present an all-optical quantitative phase microscope (QPM) design that requires no computational post-reconstruction. A follow-up literature survey suggested that the learnt architecture is similar to the generalized phase concept developed two decades ago. We then incorporate the generalized phase contrast concept to simplify the learning procedure. Furthermore, this physical optical setup is miniaturized using a diffractive deep neural network (D2NN). We outperform the existing benchmark for all-optical phase-to-intensity conversion on multiple datasets, and ours is the first demonstration of its kind on D2NNs. The proposed differentiable microscopy framework supplements the creative process of designing new optical systems and would perhaps lead to unconventional but better optical designs.

show abstract

“…7. Learned sensing: Recently, FP experimental setups were used for a "learned sensing" strategy, wherein the hardware setup itself (e.g., the illumination pattern) and the ML classification process were optimized jointly to improve algorithm performance [187][188][189]203]. This approach can reduce the required data captured in FP and help build high-speed diagnostic systems.…”

Section: Applications and Future Directionsmentioning

confidence: 99%

Fourier ptychography: current applications and future promises

et al. 2020

Self Cite

View full text Add to dashboard Cite

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

show abstract

Towards an Intelligent Microscope: Adaptively Learned Illumination for Optimal Sample Classification

Cited by 10 publications

References 15 publications

Tunable SIM: observation at varying spatiotemporal resolutions across the FOV

Tunable SIM: observation at varying spatiotemporal resolutions across the FOV

Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

Fourier ptychography: current applications and future promises

Contact Info

Product

Resources

About