Analysis of preferential localization of certain genes within the cell nuclei is emerging as a new technique for the diagnosis of breast cancer. Quantitation requires accurate segmentation of 100-200 cell nuclei in each tissue section to draw a statistically significant result. Thus, for large-scale analysis, manual processing is too time consuming and subjective. Fortuitously, acquired images generally contain many more nuclei than are needed for analysis. Therefore, we developed an integrated workflow that selects, following automatic segmentation, a subpopulation of accurately delineated nuclei for positioning of fluorescence in situ hybridization-labeled genes of interest. Segmentation was performed by a multistage watershed-based algorithm and screening by an artificial neural network-based pattern recognition engine. The performance of the workflow was quantified in terms of the fraction of automatically selected nuclei that were visually confirmed as well segmented and by the boundary accuracy of the well-segmented nuclei relative to a 2D dynamic programming-based reference segmentation method. Application of the method was demonstrated for discriminating normal and cancerous breast tissue sections based on the differential positioning of the HES5 gene. Automatic results agreed with manual analysis in 11 out of 14 cancers, all four normal cases, and all five noncancerous breast disease cases, thus showing the accuracy and robustness of the proposed approach. Published 2012 Wiley-Periodicals, Inc. y

Accurate segmentation of cell nuclei in microscope images of tissue sections is a key step in a number of biological and clinical applications. Often such applications require analysis of large image datasets for which manual segmentation becomes subjective and time consuming. Hence automation of the segmentation steps using fast, robust and accurate image analysis and pattern classification techniques is necessary for high throughput processing of such datasets. We describe a supervised learning framework, based on artificial neural networks (ANNs), to identify well-segmented nuclei in tissue sections from a multistage watershed segmentation algorithm. The successful automation was demonstrated by screening over 1400 well segmented nuclei from 9 datasets of human breast tissue section images and comparing the results to a previously used stacked classifier based analysis framework.

No abstract