Automatic granular and spinous epidermal cell identification and analysis on in vivo reflectance confocal microscopy images using cell morphological features

Abstract

Significance: Reflectance confocal microscopy (RCM) allows for real-time in vivo visualization of the skin at the cellular level. The study of RCM images provides information on the structural properties of the epidermis. These may change in each layer of the epidermis, depending on the subject's age and the presence of certain dermatological conditions. Studying RCM images requires manual identification of cells to derive these properties, which is time consuming and subject to human error, highlighting the need for an automated cell identification method.Aim: We aim to design an automated pipeline for the analysis of the structure of the epidermis from RCM images of the Stratum granulosum and Stratum spinosum.Approach: We identified the region of interest containing the epidermal cells and the individual cells in the segmented tissue area using tubeness filters to highlight membranes. We used prior biological knowledge on cell size to process the resulting detected cells, removing cells that were too small and reapplying the used filters locally on detected regions that were too big to be considered a single cell. The proposed full image analysis pipeline (FIAP) was compared with machine learning-based approaches (cell cutter, different U-Net configurations, and loss functions).Results: All methods were evaluated both on simulated data (four images) and on manually annotated RCM data (seven images). Accuracy was measured using recall and precision metrics. Both accuracy metrics were higher in the proposed FIAP for both real (precision ¼ 0.720 AE 0.068, recall ¼ 0.850 AE 0.11) and synthetic images (precision ¼ 0.835 AE 0.067, recall ¼ 0.925 AE 0.012). The tested machine learning methods failed to identify and segment keratinocytes on RCM images with a satisfactory accuracy. Conclusions:We showed that automatic cell segmentation can be achieved using a pipeline based on membrane detection, with an accuracy that matches expert manual cell identification. To our knowledge, this is the first method based on membrane detection to study healthy skin using RCM images evaluated against manually identified cell positions.

DermoGAN: multi-task cycle generative adversarial networks for unsupervised automatic cell identification on in-vivo reflectance confocal microscopy images of the human epidermis

“…A region of interest (ROI) mask was generated for and applied to all images used across all six tested methods. The ROI was identified by distinguishing the tissue from the dark background, due to the skin micro-relief lines, using a morphological-geodesic-active-contour, and removing non-informative areas in the tissue, due to low contrast and a drop in the signal-to-noise ratio, through a texture classification with a support vector machine on four features of the gray level co-occurrence matrix (homogeneity, contrast, dissimilarity, and energy 8 ).…”

“…Image classification in one of the four epidermal layers was obtained using a hybrid deep learning algorithm, 27 allowing to focus only on images of the stratum granulosum (SG) and stratum spinosum (SS), where keratinocytes are visible and identifiable on RCM images and arranged in a honeycomb pattern 28 inside of islands surrounded by dark grooves representing micro-relief lines. 8 , 29 …”

“…Noise removal was performed using a first cycle-GAN, learning the translation between RCM and binary images obtained by simulating the structure of keratinocytes 8 (described in Sec. 3.3.1 ), whereas membrane identification was performed by learning the mapping between binary images and Gabor-filtered RCM images, i.e., where membranes were highlighted.…”

“…Although automated methods have been developed to identify some lesions on RCM images, 3 – 7 most of the analysis is performed manually, which is time-consuming and subject to inter and intra-expert interpretation. 8 Hence, an automated method for keratinocyte identification on RCM images is needed, which would allow for a more reproducible, unbiased, and precise analysis. Unfortunately, image quality, heterogeneity, and low signal-to-noise ratio are a hurdle to automated methods’ development.…”

“…We compare the proposed method with seven other approaches, namely, a supervised method based on a U-net architecture, 16 a pre-trained StarDist 17 applied to Gabor-filtered images, two unsupervised approaches based on a cycle-GAN with different inputs, a tailored pipeline based on the detection of membrane morphological features, 8 the CellPose 18 algorithm for cellular segmentation, and finally DermoGAN followed by postprocessing using CellPose.…”