Mode conversion of qOBM (quantitative oblique back-illumination microscopy) stain-free tissue images to emulate H and E histology via deep learning

Abstract

Slide-free microscopy techniques have been proposed for accelerating standard histopathology and intraoperative guidance. One such technology is quantitative oblique back-illumination microscopy (qOBM), which enables real-time, label-free quantitative phase imaging of thick, unsectioned in-vivo and ex-vivo tissues. However, the grayscale phase contrast provided by qOBM differs from the colored histology images familiar to pathologists and clinicians, limiting its current potential for adoption. Here we demonstrate the application of unsupervised deep learning using a Cycleconsistent Generative Adversarial Network (CycleGAN) model to transform qOBM images into virtual hematoxylin and eosin (H&E)-stained images. The models were trained on a dataset of qOBM and H&E images of similar regions in excised brain tissue from a 9L gliosarcoma rat tumor model. We observed successful qOBM-to-H&E conversion of both uninvolved and tumor-containing specimens, as demonstrated by a classifier test. We describe several crucial preprocessing steps that improve the quality of conversion, such as intensity inversion, pixel harmonization, and color normalization. This unsupervised deep learning framework does exhibit occasional subpar performance; for example, as with GANs in general, it can create so-called "hallucinations", displaying features not actually present in the original qOBM images. We anticipate that this behavior can be minimized with more extensive training and deployment of advanced ML techniques, and that virtual-H&E-converted qOBM imaging will prove safe and appropriate for rapid tissue imaging applications.

Pilot study of fluorescence imitating brightfield imaging for rapid, slide‐free dermatopathology

“…Additionally, the images were gamma‐adjusted in GIMP to improve the appearance of otherwise too light or dark regions. To improve visual similarity to traditional H&E coloration for ease of interpretation, we developed a deep‐learning CycleGAN mode‐mapping model, similar to the MUSE‐to‐H&E conversion described in Abraham et al 12 For each case presented, multiple specimens were obtained and imaged with FIBI. One of the images is designated as the “test set,” while the other images are used as a “training set” for training the CycleGAN to perform color mapping.…”

“…For example, for Case 5 training set (squamous cell carcinoma), most patches of a large keratin pearl were removed so as not to overwhelm the training process and introduce incorrect biases. For the test set, the model is applied to overlapping patches stitched into a full montage as described in Abraham et al 12 Unless otherwise described, the presented FIBI images were color mapped using this CycleGAN model.…”