Noise Learning of Instruments for High-contrast, High-resolution and Fast Hyperspectral Microscopy and Nanoscopy

Abstract

Raman spectroscopy provides molecular fingerprint information of materials and live-cells in a label-free way, but the intrinsic low Raman scattering efficiency makes it vulnerable to noise. There has to be a trade-off among signal-to-noise ratio (SNR), imaging speed, and spatial and spectral resolutions when Raman spectroscopy is combined with microscopy and especially nanoscopy. Here, we report a noise learning (NL) approach that can fit the intrinsic noise distribution of each instrument by statistically learning the noise in the frequency domain. The fitted noise is then removed from the noisy spectra to improve their SNR as well as the contrast of hyperspectral images. The approach enhances the SNR by ca. 10 folds on a 12,500-spectra dataset, and suppresses the mean-square error by almost 150 folds. It significantly reduces the pixel-dwell time by 10 folds for tip-enhanced Raman imaging and the impact of drift on nanoimaging, leading to ca.2-fold improvement of the spatial resolution of 2.9 nm in air that reveals atomic-scale properties of bimetallic catalysts. We further demonstrate that NL can be broadly applied to enhance SNR in fluorescence and photoluminescence imaging, which allows the reduction of laser fluence by ca. 40 folds, thereby, circumventing the photobleaching and phototoxicity problems for long-term imaging of live-cells. NL manages the ground truth spectra and the instrumental noise simultaneously within the training dataset, and thus, bypasses the tedious labelling of the huge dataset required in conventional deep learning, which shifts deep learning from sample-dependent to instrument-dependent. NL is robust for a broad range of spectroscopic techniques for biomedical, photonic, and catalytic applications.

Artificial Intelligence for Surface‐Enhanced Raman Spectroscopy

“…Early disease diagnosis SVM [ 142] Low SNR spectra Predict high SNR spectra for nanometer-resolution hyperspectral imaging U-net [ 143] Single-frame Raman imaging with nonchirped femtosecond pulses Predict subcellular organelle maps DenseNet [ 144] GPR-Gaussian process regression; RR-ridge regression; However, the solution of Maxwell's equations at each spatial location and each wavelength would take extensively long solution times and large computational domains, and even cannot reach a convergence in case of some highly complicated structures. Moreover, numerical simulations require a predesign of the structures, in other words, cannot complete inverse tasks.…”

“…This method far surpassed conventional supervised learning algorithms, tackled the trade‐off between imaging resolution and speed, and also showed the potential of improving the imaging speed and minimize the possibility of photobleaching and phototoxicity caused by high incident power and long acquisition time. [ 143 ] Similarly, Horgan et al applied a DL‐enabled Raman spectroscopy for high‐throughput molecular imaging. [ 295 ] Another DenseNet based DL architecture, termed as DeepChem, was proposed in order to further map the different subcellular organelles according to the spectral features, enabling continuous imaging of the same field of view for the study of biological interaction and cooperation.…”