Evaluation of Deep Learning Architectures for Complex Immunofluorescence Nuclear Image Segmentation

Abstract

Separating and labeling each nuclear instance (instance-aware segmentation) is the key challenge in nuclear image segmentation. Deep Convolutional Neural Networks have been demonstrated to solve nuclear image segmentation tasks across different imaging modalities, but a systematic comparison on complex immunofluorescence images has not been performed. Deep learning based segmentation requires annotated datasets for training, but annotated fluorescence nuclear image datasets are rare and of limited size and complexity. In this work, we evaluate and compare the segmentation effectiveness of multiple deep learning architectures (U-Net, U-Net ResNet, Cellpose, Mask R-CNN, KG instance segmentation) and two conventional algorithms (Iterative h-min based watershed, Attributed relational graphs) on complex fluorescence nuclear images of various types.We propose and evaluate a novel strategy to create artificial images to extend the training set. Results show that instanceaware segmentation architectures and Cellpose outperform the U-Net architectures and conventional methods on complex images in terms of F1 scores, while the U-Net architectures achieve overall higher mean Dice scores. Training with additional artificially generated images improves recall and F1 scores for complex images, thereby leading to top F1 scores for three out of five sample preparation types. Mask R-CNN trained on artificial images achieves the overall highest F1 score on complex images of similar conditions to the training set images while Cellpose achieves the overall highest F1 score on complex images of new imaging conditions. We provide quantitative results demonstrat-Manuscript resubmitted on February 13, 2021.

MICRA-Net: MICRoscopy Analysis Neural Network to solve detection, classification, and segmentation from a single simple auxiliary task

“…Cell counting and segmentation is a common challenge in high-throughput analysis of optical microscopy images [8, 9, 12, 33, 34]. Both fully- and weakly-supervised DL approaches were shown to be very powerful to assess these tasks on multiple cell lines [7, 25].…”

Benchmarking of deep learning algorithms for 3D instance segmentation of confocal image datasets

“…This Unet extracts cell boundaries that are processed using 3D watershed along with conditional random fields ( a prediction concept which uses contextual information from previous labels) . (Kromp et al, 2019) and the Mask RCNN architecture was found to outperform the UNet based models in the 2D segmentation task. It may be noted that for evaluation (Zaki et al, 2020) use F1-score, while (Kromp et al, 2019) use under/oversegmentation and aggregated Jaccard index.…”

Application of Convolutional Neural Networks Towards Nuclei Segmentation in Localization-based Super-Resolution Fluorescence Microscopy Images

“…Nuclei size often varies signi cantly (from a few microns to tens of microns) among different cell types, cultured cells and tissues. Scale makes a signi cant difference in segmentation accuracy [2,15]. Networks trained on a limited dataset without scale variation or image augmentation, may get locked into recognizing only a limited set of nuclear sizes, resulting in over or under segmentation of instances as well as increased false negatives and splits and merges when applied to a test set containing different nuclear sizes.…”

“…Additionally, it suffers from individual user bias and lack of reproducibility. Convolutional neural networks (CNNs) have been adapted for nuclei segmentation in wide-eld uorescence images and bright-eld histology images with great success [1][2][3].…”