The task of segmenting cell nuclei and cytoplasm in pap smear images is one of the most challenging tasks in automated cervix cytological analysis due to specifically the presence of overlapping cells. This paper introduces a multi-pass fast watershed-based method (MPFW) to segment both nucleus and cytoplasm from large cell masses of overlapping cervical cells in three watershed passes. The first pass locates the nuclei with barrier-based watershed on the gradient-based edge map of a pre-processed image. The next pass segments the isolated, touching, and partially overlapping cells with a watershed transform adapted to the cell shape and location. The final pass introduces mutual iterative watersheds separately applied to each nucleus in the largely overlapping clusters to estimate the cell shape. In MPFW, the line-shaped contours of the watershed cells are deformed with ellipse fitting and contour adjustment to give a better representation of cell shapes. The performance of the proposed method has been evaluated using synthetic, real extended depth-of-field, and multi-layers cervical cytology images provided by the first and second overlapping cervical cytology image segmentation challenges in ISBI 2014 and ISBI 2015. The experimental results demonstrate superior performance of the proposed MPFW in terms of segmentation accuracy, detection rate, and time complexity, compared with recent peer methods.

The task of segmenting nuclei and cytoplasm in Papanicolau smear images is one of the most challenging tasks in automated cervix cytological analysis owing to the high degree of overlapping, the multiform shape of the cells and their complex structures resulting from inconsistent staining, poor contrast, and the presence of inflammatory cells. This article presents a robust continuous variational segmentation framework based on convolutional neutral network and a learned shape prior enabling an accurate analysis of overlapping cervical mass.The shape prior is dynamically modelled during the segmentation process as a weighted linear combination of shape templates from an over-complete shape dictionary under sparsity constraints. We provide quantitative and qualitative assessment of the proposed method using two databases of 153 cervical cytology images, with 870 cells in total, synthesised by accumulating real isolated cervical cells to generate overlapping cellular masses with a varying number of cells and degree of overlap. The experimental results have demonstrated that our methodology can successfully segment nuclei and cytoplasm from highly overlapping mass. Our segmentation is also competitive when compared to the * Corresponding author

No abstract

Automated segmentation of cells from cervical smears poses great challenge to biomedical image analysis because of the noisy and complex background, poor cytoplasmic contrast and the presence of fuzzy and overlapping cells. In this