UnMICST: Deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues

Abstract

Newly developed technologies have made it feasible to routinely collect highly multiplexed (20-60 channel) images at subcellular resolution from human tissues for research and diagnostic purposes. Extracting single cell data from such images requires efficient and accurate image segmentation. This starts with identification of nuclei, a challenging problem in tissue imaging that has recently benefited from the use of deep learning. In this paper, we demonstrate two generally applicable approaches to improving segmentation accuracy for multiple human tissues. The first involves the use of “real augmentations” during training. These augmentations comprise defocused and saturated image data and improve model accuracy whereas computational augmentation (Gaussian blurring) does not. The second involves collection of nuclear envelope data to better identify nuclear outlines. The two approaches cumulatively and substantially improve segmentation with three different deep learning frameworks, yielding a set of highly accurate segmentation models. We speculate that the use of real augmentations may have applications in image processing outside of microscopy.

Multi-modal digital pathology for colorectal cancer diagnosis by high-plex immunofluorescence imaging and traditional histology of the same tissue section

“…3c ). We have previously observed a similar fraction of “unidentifiable” cells even with 40-60 plex CyCIF imaging 22 and surmised that these cells are either negative for all antibody markers included in the panel or have morphologies that are difficult to segment 43 . We therefore used a previously published ML model trained on H&E data 44 to identify those cells missing labels in Orion IF images (see Methods for details of this model and its performance) and found that >50% were predicted to be smooth muscle, stromal fibroblasts, or adipocytes ( Fig.…”