UNI-EM: An Environment for Deep Neural Network-Based Automated Segmentation of Neuronal Electron Microscopic Images

Abstract

Recently, there has been rapid expansion in the field of micro-connectomics, which targets the three-dimensional (3D) reconstruction of neuronal networks from stacks of two-dimensional (2D) electron microscopy (EM) images. The spatial scale of the 3D reconstruction increases rapidly owing to deep convolutional neural networks (CNNs) that enable automated image segmentation. Several research teams have developed their own software pipelines for CNN-based segmentation. However, the complexity of such pipelines makes their use difficult even for computer experts and impossible for non-experts. In this study, we developed a new software program, called UNI-EM, for 2D and 3D CNN-based segmentation. UNI-EM is a software collection for CNN-based EM image segmentation, including ground truth generation, training, inference, postprocessing, proofreading, and visualization. UNI-EM incorporates a set of 2D CNNs, i.e., U-Net, ResNet, HighwayNet, and DenseNet. We further wrapped flood-filling networks (FFNs) as a representative 3D CNN-based neuron segmentation algorithm. The 2D- and 3D-CNNs are known to demonstrate state-of-the-art level segmentation performance. We then provided two example workflows: mitochondria segmentation using a 2D CNN and neuron segmentation using FFNs. By following these example workflows, users can benefit from CNN-based segmentation without possessing knowledge of Python programming or CNN frameworks.

Three-Dimensional Structure of Dendritic Spines Revealed by Volume Electron Microscopy Techniques

“…This raises a genuine concern about inter-animal variability and, therefore, the reproducibility of data sets. To surmount these limitations, several automated segmentation tools have recently been developed that use machine learning approaches (Arganda-Carreras et al, 2015 ; Berger et al, 2018 ; Lee et al, 2019 ; Urakubo et al, 2019 ) and the continuous refinement of segmentation tools aims to make their accuracy similar to that of a human annotator. With the future development of imaging and analysis tools, it will be possible to substantially increase the throughput of three-dimensional studies.…”

DeepMIB: User-friendly and open-source software for training of deep learning network for biological image segmentation

“…Unfortunately, in many cases, the application of these methods is not easy and requires significant knowledge in computer sciences, making it difficult to adapt by many researchers. Software developers have already started to address this challenge by developing user-friendly deep learning tools, such as Cell Profiler [ 10 ], Ilastik [ 11 ], ImageJ plug-ins DeepImageJ [ 12 ] and U-net [ 13 ], CDeep3M [ 5 ], and Uni-EM [ 14 ] that are especially suitable for biological projects. However, the overall usability is limited because they either rely on pre-trained networks without the possibility of training on new data [ 10 – 12 ], are limited to electron microscopy (EM) datasets [ 14 ], or have specialized computing requirements [ 5 , 13 ].…”

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

“…https://www.zooniverse.org/projects/h-spiers/etch-a-cell) [44]. Alternatively, computational approaches with traditional algorithms or deep learning approaches have been proposed to detect membrane neuronal and mitosis detection in breast cancer [45], mitochondria [46,47], synapses [48] and proteins [49]. Besides the well-known limitations of deep learning architectures, of significant computational power, large amount of training data and problems with unrelated datasets which show little value for unseen biological situations [50][51][52][53][54][55], the resolution of the EM data sets can enable or restrict their use for specific purposes.…”