Weakly Supervised Identification and Localization of Drug Fingerprints Based on Label-Free Hyperspectral CARS Microscopy

Abstract

Understanding drug fingerprints in complex biological samples is essential for the development of a drug. Hyperspectral coherent anti-Stokes Raman scattering (HS-CARS) microscopy, a label-free nondestructive chemical imaging technique, can profile biological samples based on their endogenous vibrational contrast. Here, we propose a deep learning-assisted HS-CARS imaging approach for the investigation of drug fingerprints and their localization at single-cell resolution. To identify and localize drug fingerprints in complex biological systems, an attentionbased deep neural network, hyperspectral attention net (HAN), was developed. By formulating the task to a multiple instance learning problem, HAN highlights informative regions through the attention mechanism when being trained on wholeimage labels. Using the proposed technique, we investigated the drug fingerprints of a hepatitis B virus therapy in murine liver tissues. With the increase in drug dosage, higher classification accuracy was observed, with an average area under the curve (AUC) of 0.942 for the high-dose group. Besides, highly informative tissue structures predicted by HAN demonstrated a high degree of similarity with the drug localization shown by the in situ hybridization staining results. These results demonstrate the potential of the proposed deep learning-assisted optical imaging technique for the label-free profiling, identification, and localization of drug fingerprints in biological samples, which can be extended to nonperturbative investigations of complex biological systems under various biological conditions.

Markedly Enhanced Analysis of Mass Spectrometry Images Using Weakly Supervised Machine Learning

“…Indeed, during the late stages of writing and proofing this manuscript, Shi et al. [ 53 ] published a similar paper demonstrating the application of a different attention‐based MIL method to hyperspectral data. In this work, they developed a method called hyperspectral attention net (HAN), which they applied to coherent anti‐Stokes Raman scattering (HS‐CARS) microscopy data.…”

Innovative Approaches for Drug Discovery: Quantifying Drug Distribution and Response with Raman Imaging

“…Shi et al used hyperspectral-CARS microscopy in conjunction with a weakly supervised deep learning model to investigate the effects of an antisense oligonucleotide drug, bepirovirsen, in murine cells. 160 They developed a hyperspectral attention net (HAN) to identify informative spatial regions at a subcellular level in tissue imaging data. HAN highlights the spatial areas indicating druginduced changes, as seen in Figure 5d.i.…”

“…To better leverage these subtle changes for drug studies, new efforts have been made to apply more sophisticated deep-learning approaches to understand the spectral change at a subcellular level after drug treatment. Shi et al used hyperspectral-CARS microscopy in conjunction with a weakly supervised deep learning model to investigate the effects of an antisense oligonucleotide drug, bepirovirsen, in murine cells . They developed a hyperspectral attention net (HAN) to identify informative spatial regions at a subcellular level in tissue imaging data.…”

Label-free Raman imaging for screening of anti-inflammatory function food

“…A real-time, visible monitoring technique that provides a comprehensive evaluation index of both pharmacodynamic and mechanism information would be highly desirable for development ( Tan et al, 2022 ; D. Zhang et al, 2014 ). Coherent Raman scattering (CRS) microscopy provides feasibility for noninvasive label-free visualization of endogenous biomolecules in biological samples, by detecting vibrations of chemical bonds within molecules ( Shi et al, 2023 ). Unlike the spontaneous Raman scattering, CRS utilizes a nonlinear process to enhance the Raman signal for fast imaging.…”