Improved BlendMask: Nuclei instance segmentation for medical microscopy images

Abstract

Nuclei instance segmentation is an important task in medical image analysis involving cell‐level pathological analysis, which is of great significance for many biomedical applications. Nuclei segmentation is a challenging task due to edge adhesions and the distribution of numerous tiny dense nuclei. In this work, a nuclei instance segmentation framework, namely, the improved BlendMask is proposed. In this framework, in order to improve the performance of detection and segmentation of dense small objects and adhering nuclei, two components, including dilated convolution aggregation module (DCA) and context information aggregation module (CIA), are designed. The DCA constructs multi‐path parallel dilated convolution, which greatly increases the receptive field of the network and the ability to capture multi‐scale contextual information. The CIA reduces the information loss in the channel by endowing the network with high‐level multi‐scale spatial context information. In addition, a novel distributional ranking loss function is given that can effectively alleviate the imbalance between the target and the background. The proposed method is validated on the DSB2018 dataset. Compared to BlendMask, this network improves by 3.6% on AP segmentation metric, and the segmentation performance of this network is superior to that of several recent classic open‐source nuclei instance segmentation methods.