Image reconstruction through a multimode fiber with a simple neural network architecture

Zhu, Changyan; Chan, Eng Aik; Wang, You; Peng, Weina; Guo, Rui; Zhang, Baile; Soci, Cesare; Chong, Yidong

doi:10.1038/s41598-020-79646-8

Cited by 71 publications

(35 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Principles. The light fields in an MMF can be resolved into a set of orthogonal spatial modes 23 that enable the transmission of spatial information. It has been verified that the information of images with 4N resolvable features can be carried in a single MMF, where N is the number of spatial modes per polarization 24 .…”

Section: Resultsmentioning

confidence: 99%

“…The performance of our demonstrated proof-of-principle system may be further improved. The wavelength of the source used here (1064 nm) is much longer than those adopted in most previous studies 17,23,29,30 , which will result in a much smaller number of excited modes and, thus, much less spatial information carried in the MMF. Hence, by using an MMF with a larger core and higher NA, more spatial information can be collected, and the resolution of the images that can be recovered will, in theory, be much higher.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

All-fiber ultrafast image detection enabled by deep learning

Liu¹,

Wang²,

Yuan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Detection of dynamical scenes at ultrafast speeds serves as a foundation for modern engineering, chemistry, material science and biomedicine, et al. For biomedical applications, in vivo microscopic imaging is often required, which has led to the development of fiber-probe-based endoscopy. However, the combination of ultrafast image acquiring with fiber endoscopy has not yet been achieved, which is vital for exploration of transient biomedical phenomena. Here, we propose a scheme of all-fiber image detection at an ultrafast speed without any free-space optical elements. Image detection is achieved based on the transformation of two-dimensional spatial information into one-dimensional temporal pulsed signal streams by leveraging the high intermodal dispersion in a multimode fiber. Deep learning algorithms are subsequently deployed to reconstruct the images detected by the fiber probe from the temporal waveforms acquired at the other end of the fiber. The fiber probe can directly detect micron-scale objects without any bulk objective, and image detection has been experimentally realized with a high frame rate (15 Mfps), a large frame depth (104) and extremely short shutter time (30 ps) simultaneously. This ultrafast detection scheme, combined with high mechanical flexibility and high level of integration, can stimulate future research on exploring various in-vivo ultrafast phenomena.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

All-fiber ultrafast image detection enabled by deep learning

Liu¹,

Wang²,

Yuan³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This dataset is fed into a neural network, and the predicted image is obtained by just analyzing the front speckle reflection of the MMF. For a simpler approach, Zhu et al [82] use a single hidden layer dense neural network (SHL-DNN) to reconstruct images utilizing speckle patterns from the surface of MMF. The use of SHL-DNN generates similar results as U-Net, a complex convolutional neural network (CNN) originally developed for biomedical imaging, with substantially less training time and network complexity.…”

Section: Artificial Intelligence Techniquementioning

confidence: 99%

Optical-Mechanical Configuration of Imaging Operation for Endoscopic Scanner: A Review

Leong

Mokhtar²,

Zan

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Miniaturized endoscopic scanners have had a significant impact on high-resolution optical imaging. Technological advancements in micro-electromechanical systems and optical fiber technology have resulted in various optical-mechanical configurations designed to fulfill specific requirements. However, it is still challenging to provide comprehensive, undistorted images with high-resolution images of target samples. This paper reviews the optical imaging techniques utilized in cantilever-based endoscopic scanners by analyzing and comparing their key performances, pros and cons, and corresponding optical components needed to develop the system. The concept of multimodal imaging is then highlighted by discussing its principle and current status in endoscopic scanners. We also reviewed the scanning configurations concerning their mechanical components, general structures, and drive signals for different scanning patterns. The feedback control aspect in endoscopic scanners is then addressed, with a discussion of its role in mitigating undesired nonlinear vibration effects and a survey of current implementations. Finally, we discuss the current and potential applications of endoscopic scanners, artificial intelligence techniques in image reconstruction, and disease detection and provide recommendations on endoscopic scanner system design for future reference.

show abstract

“…Recently, neural networks have attracted increasing attention in the optical community, allowing for the reconstruction of input information after propagation through random complex media. [ 17–28 ] In fibers, convolutional neural networks (CNN) were shown to produce reconstructions with a similar fidelity to the TM approach. [ 18,19,21 ] Most previous works were limited to a single, static, fiber conformation.…”

Section: Introductionmentioning

confidence: 99%

Image Transmission Through a Dynamically Perturbed Multimode Fiber by Deep Learning

Resisi

Popoff

Bromberg

2021

Laser & Photonics Reviews

View full text Add to dashboard Cite

When multimode optical fibers are perturbed, the data that is transmitted through them is scrambled. This presents a major difficulty for many possible applications, such as multimode fiber based telecommunication and endoscopy. To overcome this challenge, a deep learning approach that generalizes over mechanical perturbations is presented. Using this approach, successful reconstruction of the input images from intensity‐only measurements of speckle patterns at the output of a 1.5 m‐long randomly perturbed multimode fiber is demonstrated. The model's success is explained by hidden correlations in the speckle of random fiber conformations.

show abstract

Image reconstruction through a multimode fiber with a simple neural network architecture

Cited by 71 publications

References 44 publications

All-fiber ultrafast image detection enabled by deep learning

All-fiber ultrafast image detection enabled by deep learning

Optical-Mechanical Configuration of Imaging Operation for Endoscopic Scanner: A Review

Image Transmission Through a Dynamically Perturbed Multimode Fiber by Deep Learning

Contact Info

Product

Resources

About