Nuclei counting in microscopy images with three dimensional generative adversarial networks

Abstract

Microscopy image analysis can provide substantial information for clinical study and understanding of biological structures. Two-photon microscopy is a type of fluorescence microscopy that can image deep into tissue with nearinfrared excitation light. We are interested in methods that can detect and characterize nuclei in 3D fluorescence microscopy image volumes. In general, several challenges exist for counting nuclei in 3D image volumes. These include crowding and touching of nuclei, overlapping of nuclei, and shape and size variances of the nuclei. In this paper, a 3D nuclei counter using two different generative adversarial networks (GAN) is proposed and evaluated. Synthetic data that resembles real microscopy image is generated with a GAN and used to train another 3D GAN that counts the number of nuclei. Our approach is evaluated with respect to the number of groundtruth nuclei and compared with common ways of counting used in the biological research. Fluorescence microscopy 3D image volumes of rat kidneys are used to test our 3D nuclei counter. The accuracy results of proposed nuclei counter are compared with the ImageJ's 3D object counter (JACoP) and the 3D watershed. Both the counting accuracy and the object-based evaluation show that the proposed technique is successful for counting nuclei in 3D.

NISNet3D: Three-Dimensional Nuclear Synthesis and Instance Segmentation for Fluorescence Microscopy Images

“…In other words, the voxel in Î vec can be a negative number or a large number. Unlike previous methods [31, 40, 41] that directly learn the distance transform map, the 3D vector field volume contains both nuclei centroid and boundary information which can avoid over-detection for irregular nuclei. The output 3D vector field volumes Î vec are compared with the ground truth vector field volume I vec and optimized using the Mean Square Error (MSE) loss function, whereas the segmentation results I mask are compared with the ground truth binary volumes I bi and optimized using the combination of Focal Loss [42] ℒ FL and Tversky Loss [43] ℒ TL .…”

“…In other words, the entries in Î vec can be negative numbers or large numbers. Unlike previous methods [31, 51, 52] that directly learn the distance transform map, the 3D vector field array contains both the distance and direction of the nearest nuclei centroid from the current voxel location. This can help NISNet3D avoid the multiple detection of irregular shaped nuclei.…”

Applying deep learning to melanocyte counting on fluorescent TRP1 labelled images of in vitro skin model

“…The first approaches involved image processing methods based on color and texture descriptors combined with morphology filtering (Berge et al, 2011;Chadha et al, 2020). However, over the last decade, the state of the art has been achieved by deep learning based approaches, and more precisely by Convolutional Neural Networks (CNNs) (Khan et al, 2016;Cohen et al, 2017;Xie et al, 2018a;Falk et al, 2019;Han et al, 2019;Xie et al, 2018b;Paulauskaite-Taraseviciene et al, 2019;Liu et al, 2019;Sierra et al, 2020;Zheng et al, 2020;He et al, 2021). In object counting, it is common to distinguish between detection-based methods (Arteta et al, 2016;Laradji et al, 2018;Xie et al, 2018b;Falk et al, 2019), which are designed to precisely locate objects before counting them, and regression-based methods (Cohen et al, 2017;Xie et al, 2018a;He et al, 2021), which directly output a number of cells without necessarily detecting their precise locations.…”

Implicit Neural Representations for Generative Modeling of Living Cell Shapes

“…This fuels an interest in methods for synthesis of microscopy images and accompanying ground truth masks. The synthesis of microscopy images from ground truth masks has been widely studied [5], and has taken a major leap with the advent of generative adversarial networks [6][7][8][9][10][11][12]. In this work, we focus on the synthesis of ground truth masks.…”

“…Here, a key question is how cell shapes should be represented. A range of parametric models have been proposed that use ellipses [6] or elliptical Fourier descriptors [13] in 2D, statistical shape models [8] in 2D+time, ellipsoids [11,12] (3D) and spherical harmonics [14] in 3D, or ellipsoids deformed using active contours in 3D+time [5]. Deep learning has led to the popularization of volumetric voxel-based representations in 3D due to their natural integration with CNN architectures [9,10,15].…”