Deep learning-based method for segmenting epithelial layer of tubules in histopathological images of testicular tissue

Abstract

Purpose: There is growing concern that male reproduction is affected by environmental chemicals. One way to determine the adverse effect of environmental pollutants is to use wild animals as monitors and evaluate testicular toxicity using histopathology. We propose an automated method to process histology images of testicular tissue.Approach: Testicular tissue consists of seminiferous tubules. Segmenting the epithelial layer of the seminiferous tubule is a prerequisite for developing automated methods to detect abnormalities in tissue. We suggest an encoder-decoder fully connected convolutional neural network model to segment the epithelial layer of the seminiferous tubules in histological images. The ResNet-34 is used in the feature encoder module, and the squeeze and excitation attention block is integrated into the encoding module improving the segmentation and localization of epithelium.Results: We applied the proposed method for the two-class problem, where the epithelial layer of the tubule is the target class. The F-score and Intersection over Union of the proposed method are 0.85 and 0.92. Although the proposed method is trained on a limited training set, it performs well on an independent dataset and outperforms other state-of-the-art methods. Conclusion:The pretrained ResNet-34 in the encoder and attention block suggested in the decoder result in better segmentation and generalization. The proposed method can be applied to testicular tissue images from any mammalian species and can be used as the first part of a fully automated testicular tissue processing pipeline. The dataset and codes are publicly available on GitHub.