Deep learning-enhanced light-field imaging with continuous validation

Wagner, Nils; Beuttenmueller, Fynn; Norlin, Nils; Gierten, Jakob; Wittbrodt, Joachim; Weigert, Martin; Hufnagel, Lars; Prevedel, Robert; Kreshuk, Anna

doi:10.1101/2020.07.30.228924

Cited by 3 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Light field microscopy is able to capture 3D spatial information in a single camera frame, allowing almost instantaneous 3D imaging. This high-speed capability promoted the realistic applications of light field microscopy in biology and neurobiology providing the visualization of cardiovascular dynamics, the reduction of image reconstruction artifacts, and the recording of brain neuronal activity, for example [158]. Saliency object detection (SOD) aims to mimic the HVS to detect objects or areas that attract human attention.…”

Section: Other Light Field Imaging Applicationsmentioning

confidence: 99%

Learning-based light field imaging: an overview

Mahmoudpour,

Pagliari,

Schelkens

2024

J Image Video Proc.

View full text Add to dashboard Cite

Conventional photography can only provide a two-dimensional image of the scene, whereas emerging imaging modalities such as light field enable the representation of higher dimensional visual information by capturing light rays from different directions. Light fields provide immersive experiences, a sense of presence in the scene, and can enhance different vision tasks. Hence, research into light field processing methods has become increasingly popular. It does, however, come at the cost of higher data volume and computational complexity. With the growing deployment of machine-learning and deep architectures in image processing applications, a paradigm shift toward learning-based approaches has also been observed in the design of light field processing methods. Various learning-based approaches are developed to process the high volume of light field data efficiently for different vision tasks while improving performance. Taking into account the diversity of light field vision tasks and the deployed learning-based frameworks, it is necessary to survey the scattered learning-based works in the domain to gain insight into the current trends and challenges. This paper aims to review the existing learning-based solutions for light field imaging and to summarize the most promising frameworks. Moreover, evaluation methods and available light field datasets are highlighted. Lastly, the review concludes with a brief outlook for future research directions.

show abstract

Section: Other Light Field Imaging Applicationsmentioning

confidence: 99%

Learning-based light field imaging: an overview

Mahmoudpour,

Pagliari,

Schelkens

2024

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Using sophisticated custom-built optical setups, functional 4D activity mapping in Caenorhabditis elegans, zebrafish, and primary neuron cultures have been performed at scanning rates between 0.8 to 50 Hz [31][32][33][34][35][36][37][38]. Moreover, artificial intelligenceenhanced microscopy has improved the performance of a light-field microscope in functional imaging of neuronal activity of the zebrafish brain [39]. While these setups are in principle suited to perform volumetric imaging of neuronal networks, they require advanced optical expertise in their assembly and alignment [39][40][41][42][43][44][45][46][47][48][49].…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, artificial intelligenceenhanced microscopy has improved the performance of a light-field microscope in functional imaging of neuronal activity of the zebrafish brain [39]. While these setups are in principle suited to perform volumetric imaging of neuronal networks, they require advanced optical expertise in their assembly and alignment [39][40][41][42][43][44][45][46][47][48][49]. So far, few of these systems have been employed to study brain and retinal organoids [6].…”

Section: Introductionmentioning

confidence: 99%

A minimal-complexity light-sheet microscope maps network activity in 3D neuronal systems

Wysmolek

Kiessler

Salbaum

et al. 2022

Preprint

View full text Add to dashboard Cite

In vitro systems mimicking brain regions, such as brain organoids, are revolutionizing the field of neuroscience. However, characterization of their electrical activity has remained a challenge as this requires electrophysiological readout at millisecond timescale in 3D at single-neuron resolution. While custom-built microscopes used with genetically encoded sensors start to grant this access, a thorough 3D characterization of organoid neural activity has not been performed yet, limited by the combined complexity of the optical and the biological system. Here, we introduce a simple microscope for volumetric characterization of network activity with single-neuron resolution. To provide a versatile, accessible platform to the neuroscience community, we designed a minimalistic light-sheet microscope tai-lored to computational neuroscience tools to extract calcium traces. As a proof of principle, we created a 3D connectivity map of the neuronal network by imaging its spontaneous activity. High performance, low complexity setups such as ours will empower researchers to study the formation of neuronal networks in vitro for fundamental and neurodegeneration research.

show abstract

“…Deep learning technology has become increasingly popular in the realm of microscopic imaging, with a broad range of applications including denoising 7,24 , modal conversion 25,26 , resolution enhancement 27 , downstream task analysis 28,29 , and high-speed imaging [30][31][32] of three-color labeled cell samples with low rates of molecular misidentification 33 . However, this approach was specifically designed for classifying different dyes and cannot be applied to identify the continuous spectral changes in different environments.…”

mentioning

confidence: 99%

Deep learning-enhanced single-molecule spectrum imaging

Hao

Zhang

et al. 2023

Preprint

View full text Add to dashboard Cite

Fluorescence is widely used in biological imaging and biosensing. Rich information can be revealed from the fluorescence spectrum of fluorescent molecules, such as pH, viscosity and polarity of the molecule’s environment, and distance between two FRET molecules. However, constructing the fluorescence spectrum of a single fluorescent molecule typically requires a significant number of photons, which can suffer from photobleaching and therefore limit its potential applications. Here we propose a deep learning-enhanced single-molecule spectrum imaging method (SpecGAN) for improving the single-molecule spectrum imaging efficiency. In SpecGAN, the photon flux required to extract a single-molecule fluorescence spectrum can be reduced by 100 times, which enables it two orders of magnitude higher temporal resolution compared to the conventional single-molecule spectrometer. The concept of SpecGAN was validated through numerical simulation and single Nile Red molecule spectrum imaging on support lipid bilayers (SLBs). With SpecGAN, the super-resolution spectrum image of the COS-7 membrane can be reconstructed with merely 12,000 frames of single-molecule localization images, which is almost half of the previously reported frame count for spectrally resolved super-resolution imaging. The low photon flux requirement and high temporal resolution of SpecGAN make it a promising tool for investigating the molecular spectrum dynamics related to biological functions or biomolecule interactions.

show abstract

Deep learning-enhanced light-field imaging with continuous validation

Cited by 3 publications

References 32 publications

Learning-based light field imaging: an overview

Learning-based light field imaging: an overview

A minimal-complexity light-sheet microscope maps network activity in 3D neuronal systems

Deep learning-enhanced single-molecule spectrum imaging

Contact Info

Product

Resources

About