Multichannel correlation improves the noise tolerance of real-time hyperspectral microimage mosaicking

Abstract

Live-subject microscopies, including microendoscopy and other related technologies, offer promise for basic biology research as well as the optical biopsy of disease in the clinic. However, cellular resolution generally comes with the trade-off of a microscopic field-of-view. Microimage mosaicking enables stitching many small scenes together to aid visualization, quantitative interpretation, and mapping of microscale features, for example, to guide surgical intervention. The development of hyperspectral and multispectral systems for biomedical applications provides motivation for adapting mosaicking algorithms to process a number of simultaneous spectral channels. We present an algorithm that mosaics multichannel video by correlating channels of consecutive frames as a basis for efficiently calculating image alignments. We characterize the noise tolerance of the algorithm by using simulated video with known ground-truth alignments to quantify mosaicking accuracy and speed, showing that multiplexed molecular imaging enhances mosaic accuracy by leveraging observations of distinct molecular constituents to inform frame alignment. A simple mathematical model is introduced to characterize the noise suppression provided by a given group of spectral channels, thus predicting the performance of selected subsets of data channels in order to balance mosaic computation accuracy and speed. The characteristic noise tolerance of a given number of channels is shown to improve through selection of an optimal subset of channels that maximizes this model. We also demonstrate that the multichannel algorithm produces higher quality mosaics than the analogous single-channel methods in an empirical test case. To compensate for the increased data rate of hyperspectral video compared to single-channel systems, we employ parallel processing via GPUs to alleviate computational bottlenecks and to achieve real-time mosaicking even for video-rate multichannel systems anticipated in the future. This implementation paves the way for real-time multichannel mosaicking to accompany next-generation hyperspectral and multispectral video microscopy.

Application of Microspectral Imaging in Motor and Sensory Nerve Classification

“…At present, clinical methods for nerve classification mainly focus on electrophysiology and tissue characteristics [ 13 ]. There are few studies on the classification of old nerves using optical techniques.…”

Advancing towards open-source biophotonic instrumentation for multiphoton imaging and photodynamic therapy

“…3.9) dyed in sulforhodamine 101 are collected at 20 fps. The micromosaicking technique from our previous work [27] is used to combine multiple frames to increase the FOV. The image of the human epidermoid carcinoma cells (A431, ATCC)…”

“…photobleaching and phototoxicity effects thanks to the longer wavelength of the excitation light and because scattered emitted photons are collected by a wide area detector (no pinhole is needed).Additionally, the wavelength of the excitation laser source is usually far away from the fluorescence signal, thus, the unwanted laser reflection can be suppressed efficiently by a spectral filter, reducing the background noise[2,27]. These characteristic benefits make multiphoton excitation a suitable tool for biomedical imaging applications in vivo.…”

New methods and tools for optimizing translational precision photomedicine

“…This lag precludes monitoring image quality and online identification of target biological phenomenon during experiments, among other potential new applications. Moreover, subsequent bioimage informatics routines to extract molecular spatiotemporal localizations like hyperspectral image mosaicking 226,227 are also prolonged, further inhibiting efficient cycles of progressive data visualization, interpretation, and analysis. Significant effort has been dedicated to enhancing NNLS performance through algorithmic improvements 228,229 , including the fast-NNLS algorithm 228 that provides a 2× boost over standard NNLS (Fig.…”