Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images

Abstract

Background Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures. Results Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture. Conclusions We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community (https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation).

Convolutional neural network approach for the automated identification ofin cellulocrystals

“…Due to that, we chose 50 images as the best compromise, thus selecting this as a base for the following training strategies. Regarding cell recognition, the model trained with the science-bowl dataset is able to recognize the cell bodies (Figure 4) based on previous training approaches (Mela & Liu, 2021). Our results indicate that learning approaches are useful for recognizing in cellulo crystals, using a limited number of epochs and annotated images.…”

Automated segmentation of endometriosis using transfer learning technique

“…Similarly, the optimizers used for U-Net architectures were RMSprop, SGD, and Adam. 39 Also the epoch sizes were chosen based on empirical analysis. The epoch size identified was 10, 20 and 30.The results obtained are illustrated in Figures 4-6.…”

Enhanced Wind Field Spatial Downscaling Method Using UNET Architecture and Dual Cross-Attention Mechanism

“…The UNET model has shown outstanding performance in image segmentation tasks, particularly in tasks requiring precise detail segmentation. In recent years, experts have attempted to introduce it into the field of image super-resolution with good results [35][36][37][38]. The traditional UNET architecture efficiently extracts multi-scale features through design, consisting of symmetric branches for encoding and decoding.…”