Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy

Abstract

The goal of oncologic surgeries is to completely resect tumor tissue, yet in up to 40% of such surgeries, positive marginsare found in the resected tissues. Postoperative histology using H&E-stained brightfield microscopy is the gold standard for determining margin status, but rapid frozen section analysis is sometimes performed for intraoperative guidance, albeit with inaccuracies. In this work, we introduce a virtual histological imaging method based on a non-contact, reflection-mode ultraviolet photoacoustic remote sensing and scattering microscope, combined with deep learning using a cycle-consistent generative adversarial network. The system is capable of high-resolution scanning with 390 nm resolution comparable toconventional histopathology, and fast widefield scanning, generating images with histological realism in freshly-resected thick tissues or thin sections. Cytologic and architectural features of interest are readily identifiable in virtual histology images of benign and malignant tissues. To evaluate system performance, a blinded study of pathologists rating image quality metrics was conducted, with our virtual histology approach offering preferred hematoxylin-like detail (P=0.0018) and overall stain quality (P=0.0321) compared to frozen section analysis.

Virtual histological staining of unlabeled autopsy tissue

“…Hence, we anticipate that our autopsy virtual staining method can be effectively adapted to other organs with appropriate model refinement and retraining. Moreover, while our study employed autofluorescence images of unlabeled autopsy tissue sections as the input modality-chosen due to their prevalent availability and seamless integration into current clinical tissue scanners-other label-free microscopy modalities can also be exploited for virtual staining, including, e.g., quantitative phase imaging 20 , Raman microscopy 52 , nonlinear microscopy 53,54 and photoacoustic microscopy 31,55 . In addition, lung morphology can be significantly changed by the fixation method (e.g., using inflation of fixative solutions through the airways, vascular perfusion techniques, or passive fixative immersion and diffusion 56 ).…”

Photoacoustic imaging for cutaneous melanoma assessment: a comprehensive review

“…Recently, all-optical PA microscopy in reflection-mode has been shown to be able to form histology-like images of various cancers on unstained slides to distinguish tissue types 142 , 267 . The principle behind this technology is that by concurrently measuring radiative and nonradiative (in the form of acoustic) emissions from tissue samples illuminated with light in the UV range (266 nm), it is possible to differentiate DNA, RNA, collagen, and elastin, among other chromophores.…”

“…Reproduced from Ref. 142. (f) Exogenous contrast agents: types of contrast agents utilized, implemented in animal studies.…”

DeepDOF-SE: affordable deep-learning microscopy platform for slide-free histology

“…Unsupervised CycleGAN-based 32 virtual staining can be trained on unpaired images 33 , making it one of the most frequently used deep learning frameworks for different histological applications, including label-free virtual staining, stain-to-stain transformation, and correction of stain variations 31,34 . In virtual staining applications, CycleGAN was evaluated with different imaging modalities, such as MUSE and photoacoustic microscopy, and virtual staining has been demonstrated in different tissue types including brain, breast, prostate, and bone specimens 15,33,35 . In DeepDOF-SE, we demonstrate a two-step semi-supervised scheme to train the Cycle-GAN for virtual staining, generating artifact-free virtual H&E while avoiding the need for acquiring paired data.…”