Characterizing robustness and sensitivity of convolutional neural networks for quantitative analysis of mitochondrial morphology

Abstract

Background: Quantitative analysis of mitochondrial morphology plays important roles in studies of mitochondrial biology. The analysis depends critically on segmentation of mitochondria, the image analysis process of extracting mitochondrial morphology from images. The main goal of this study is to characterize the performance of convolutional neural networks (CNNs) in segmentation of mitochondria from fluorescence microscopy images. Recently, CNNs have achieved remarkable success in challenging image segmentation tasks in several disciplines. So far, however, our knowledge of their performance in segmenting biological images remains limited. In particular, we know little about their robustness, which defines their capability of segmenting biological images of different conditions, and their sensitivity, which defines their capability of detecting subtle morphological changes of biological objects. Methods: We have developed a method that uses realistic synthetic images of different conditions to characterize the robustness and sensitivity of CNNs in segmentation of mitochondria. Using this method, we compared performance of two widely adopted CNNs: the fully convolutional network (FCN) and the U-Net. We further compared the two networks against the adaptive active-mask (AAM) algorithm, a representative of high-performance conventional segmentation algorithms. Results: The FCN and the U-Net consistently outperformed the AAM in accuracy, robustness, and sensitivity, often by a significant margin. The U-Net provided overall the best performance. Conclusions: Our study demonstrates superior performance of the U-Net and the FCN in segmentation of mitochondria. It also provides quantitative measurements of the robustness and sensitivity of these networks that are essential to their applications in quantitative analysis of mitochondrial morphology.

Characterizing Robustness of Deep Neural Networks in Semantic Segmentation of Fluorescence Microscopy Images

“…So far, few studies have examined the robustness of DNNs in segmentation of FM images. In [39] an assay was developed to synthesize images to benchmark robustness of FCN [40] and U-Net [41] against three types of corruptions, namely noise, spatial invariant blurring, and spatial variant blurring, in semantic segmentation of mitochondria. However, a key limitation of the study is that the synthesized mitochondria are unrealistic.…”

“…In a previous study [39], the foreground and background of FM images of mitochondria were modeled using a Gamma distribution and a Gaussian distribution, respectively, to synthesize images from binary masks. Pixels in foreground regions defined by the binary masks were filled with random samples from the Gamma distribution [39]. This method, referred to as Random Fill, cannot capture spatial patterns of pixel intensities and diffusive boundaries of real image objects.…”

“…2). Moreover, because of the sharp boundaries of the synthetic images, DNN models trained on them tend to over segment on real images with diffusive boundaries [39].…”

Fluo-Fluo translation based on deep learning

“…Recently, convolutional neural networks (CNNs) have been proved to have unprecedented performance in microscopy image analysis tasks [4][5][6][7][8][9] . In 2018, Christiansen et al [10] used CNNs to generate virtual fluorescence images from the unlabeled transmitted light images to identify the location and texture of nuclei and membranes, the health of cells, the types of cells, and the subcellular structures.…”

Metabolic Regulation of Mitochondrial Morphologies in Pancreatic Beta Cells: Bioenergetics-Mitochondrial Dynamics Coupling

“…[24][25][26] Image preprocessing and analysis techniques are indispensable for revealing information from these data and further providing quantitative evidence of the results and discoveries based on mitochondrial dynamics. [27][28][29] In contrast, in silico models describe the relationships between substrate input and mitochondrial ATP production using computer simulations. The widely adopted [30][31][32][33][34][35] mathematical model of beta-cell mitochondria by Magnus and Kaiser [36] described the influence of intracellular calcium dynamics and adenylate levels on ATP synthesis and the mitochondrial membrane potential.…”