Deep transfer learning-based hologram classification for molecular diagnostics

Kim, Sungjin; Wang, Chuangqi; Zhao, Bing; Im, Hyungsoon; Min, Jouha; Choi, Hee June; Tadros, Joseph; Choi, Nu Ri; Castro, Cesar M.; Weissleder, Ralph; Lee, Hakho; Lee, Kwonmoo

doi:10.1101/192559

Cited by 16 publications

(24 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these learning‐based plankton classification methods cannot be applied to holographic data directly, as holographic patterns are morphologically different from microscopic images. So far, deep learning‐based hologram processing approaches have been developed by various groups for hologram generation (Horisaki et al 2018), phase retrieval, biological and medical hologram reconstruction (Wu et al 2018; Ren et al 2019; Rivenson et al 2019; Shao et al 2020), and hologram classification (Jo et al 2017; Kim et al 2018 b ; Zhang et al 2018 b ). In particular, Gӧrӧcs et al (2018) applied a deep learning method for species classification on images containing plankton obtained from an inline holographic imaging flow cytometer.…”

Section: Figurementioning

confidence: 99%

Automated plankton classification from holographic imagery with deep convolutional neural networks

Guo

Nyman

Nayak

et al. 2020

Limnology & Ocean Methods

View full text Add to dashboard Cite

In situ digital inline holography is a technique which can be used to acquire high‐resolution imagery of plankton and examine their spatial and temporal distributions within the water column in a nonintrusive manner. However, for effective expert identification of an organism from digital holographic imagery, it is necessary to apply a computationally expensive numerical reconstruction algorithm. This lengthy process inhibits real‐time monitoring of plankton distributions. Deep learning methods, such as convolutional neural networks, applied to interference patterns of different organisms from minimally processed holograms can eliminate the need for reconstruction and accomplish real‐time computation. In this article, we integrate deep learning methods with digital inline holography to create a rapid and accurate plankton classification network for 10 classes of organisms that are commonly seen in our data sets. We describe the procedure from preprocessing to classification. Our network achieves 93.8% accuracy when applied to a manually classified testing data set. Upon further application of a probability filter to eliminate false classification, the average precision and recall are 96.8% and 95.0%, respectively. Furthermore, the network was applied to 7500 in situ holograms collected at East Sound in Washington during a vertical profile to characterize depth distribution of the local diatoms. The results are in agreement with simultaneously recorded independent chlorophyll concentration depth profiles. This lightweight network exemplifies its capability for real‐time, high‐accuracy plankton classification and it has the potential to be deployed on imaging instruments for long‐term in situ plankton monitoring.

show abstract

Section: Figurementioning

confidence: 99%

Automated plankton classification from holographic imagery with deep convolutional neural networks

Guo

Nyman

Nayak

et al. 2020

Limnology & Ocean Methods

View full text Add to dashboard Cite

show abstract

“…A version currently in clinical trials is a deep‐learning–enabled fluorescence cytometer, which is a stand‐alone unit weighting approximately 6 pounds. Prototype versions of this instrumentation were originally developed for global health applications 18‐20 and are currently being adapted for high‐resolution multiplexed image cytometry. Figure 3 illustrates the FAST‐FNA pipeline technology for HNSCC, including FNAB sample collection and staining with FAST antibodies in cyclic fashion, image processing, the use of a deep‐learning algorithm, and the generation of quantitative biomarker data.…”

Section: Fast‐fna Technology and Automated Readersmentioning

confidence: 99%

New technology on the horizon: Fast analytical screening technique FNA (FAST‐FNA) enables rapid, multiplex biomarker analysis in head and neck cancers

Pai

Faquin

Sadow

et al. 2020

Cancer Cytopathology

Self Cite

View full text Add to dashboard Cite

PD‐L1 profiling was recently approved by the US Food and Drug Administration as a companion diagnostic for anti‐PD1 treatment in patients with head and neck cancer, ushering in a new era for precision medicine. However, the routine development and implementation of such testing is still limited by current clinical workflows and the lack of better and more comprehensive alternatives. In this review, the authors discuss the real‐world challenges of clinically based biomarker testing and highlight the advantages of developing fine‐needle aspiration (FNA)‐based biomarker testing that would enable frequent and serial tumor sampling. A conceptual and technological innovation is introduced, fast analytical screening technique (FAST)‐FNA (FAST chemistry‐enabled FNA), which is being developed to inform immunotherapy treatment options in patients with head and neck cancer and to assist with the development of the next generation of predictive biomarkers.

show abstract

“…Specifically, we combine two high-performance deep learning architectures, U-Net and VGG19 pretrained models [43][44][45][46]. This transfer learning approach has been widely used to reduce the size of the training set and minimize overfitting [47][48][49][50]. After the segmentation, various morphological features were extracted and unsupervised learning was applied to identify distinct subpopulations of VSMC spheroids.…”

Section: Fak Controls Cellular Processes Including Cell Adhesion and mentioning

confidence: 99%

Machine learning reveals heterogeneous responses to FAK, Rac, Rho, and Cdc42 inhibition on vascular smooth muscle cell spheroid formation and morphology

Vaidyanathan

Wang

Krajnik

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Hyper proliferation of vascular smooth muscle cells (VSMCs) contributes to neointima formation in atherosclerosis and the response to vascular injury. Understanding how to control VSMC proliferation would advance the effort to treat vascular disease. Drug responses are often different among patients with the same vascular disease condition, making it difficult to cater patientspecific treatments (existence of heterogeneity). Thus, we examined variations in response to drug treatments using VSMC spheroids that mimic vascular disease condition in vivo. FAK and its downstream Rho GTPases (Rac, Rho, and Cdc42) control cell-cell contact and play a key role in vascular pathology. Here, we tested the importance of FAK and Rho GTPases in spheroid formation. VSMC spheroids were made using a hanging-drop method with either inhibitors or vehicle control. Changes in morphology were used as a key indicator of spheroid response to drug treatment. A machine learning (ML) image segmentation (VGG16-U-net) was used to segment the spheroid images. After the morphological features were extracted from the segmented images, a two-level cluster framework was used to cluster VSMC spheroids into different morphologies. We found that FAK and Rho GTPases are required for normal spheroid formation. Next, we analyzed the various morphologies of disrupted spheroids resulting from drug treatment using our ML pipeline. The first-level clustering analysis showed the presence of 4 clusters of spheroids with rounded and disrupted morphologies. The subsequent second-level clustering analysis identified the presence of four distinct morphological clusters among these disrupted spheroids, and they exhibited differential responses to FAK and Rho GTPases inhibition. Particularly, we found FAK and Rho specific morphological phenotypes, thus suggesting that there may be two distinct pathways governing VSMC spheroid formation.Collectively, we revealed there are significant heterogeneities in the drug responses of spheroid formation, which were overlooked in previous analyses. This is our first step towards developing the ML-based method that can be used to assess the effects of different drugs on the VSMC spheroid model for better characterization of pathologic progression of vascular disease.

show abstract

Deep transfer learning-based hologram classification for molecular diagnostics

Cited by 16 publications

References 35 publications

Automated plankton classification from holographic imagery with deep convolutional neural networks

Automated plankton classification from holographic imagery with deep convolutional neural networks

New technology on the horizon: Fast analytical screening technique FNA (FAST‐FNA) enables rapid, multiplex biomarker analysis in head and neck cancers

Machine learning reveals heterogeneous responses to FAK, Rac, Rho, and Cdc42 inhibition on vascular smooth muscle cell spheroid formation and morphology

Contact Info

Product

Resources

About