BackgroundThe extraction of overlapping cell nuclei is a critical issue in automated
diagnosis systems. Due to the similarities between overlapping and malignant
nuclei, misclassification of the overlapped regions can affect the automated
systems’ final decision. In this paper, we present a method for detecting
overlapping cell nuclei in Pap smear samples.MethodJudgement about the presence of overlapping nuclei is performed in three steps
using an unsupervised clustering approach: candidate nuclei regions are located
and refined with morphological operations; key features are extracted; and
candidate nuclei regions are clustered into two groups, overlapping or
non-overlapping, A new combination of features containing two local minima-based
and three shape-dependent features are extracted for determination of the presence
or absence of overlapping. F1 score, precision, and recall values are used to
evaluate the method’s classification performance.ResultsIn order to make evaluation, we compared the segmentation results of the
proposed system with empirical contours. Experimental results indicate that
applied morphological operations can locate most of the nuclei and produces
accurate boundaries. Independent features significance test indicates that our
feature combination is significant for overlapping nuclei. Comparisons of the
classification results of a fuzzy clustering algorithm and a non-fuzzy clustering
algorithm show that the fuzzy approach would be a more convenient mechanism for
classification of overlapping.ConclusionThe main contribution of this study is the development of a decision mechanism
for identifying overlapping nuclei to further improve the extraction process with
respect to the segmentation of interregional borders, nuclei area, and radius.
Experimental results showed that our unsupervised approach with proposed feature
combination yields acceptable performance for detection of overlapping
nuclei.