U-RISC: An Annotated Ultra-High-Resolution Electron Microscopy Dataset Challenging the Existing Deep Learning Algorithms

Abstract

Connectomics is a developing field aiming at reconstructing the connection of the neural system at the nanometer scale. Computer vision technology, especially deep learning methods used in image processing, has promoted connectomic data analysis to a new era. However, the performance of the state-of-the-art (SOTA) methods still falls behind the demand of scientific research. Inspired by the success of ImageNet, we present an annotated ultra-high resolution image segmentation dataset for cell membrane (U-RISC), which is the largest cell membrane-annotated electron microscopy (EM) dataset with a resolution of 2.18 nm/pixel. Multiple iterative annotations ensured the quality of the dataset. Through an open competition, we reveal that the performance of current deep learning methods still has a considerable gap from the human level, different from ISBI 2012, on which the performance of deep learning is closer to the human level. To explore the causes of this discrepancy, we analyze the neural networks with a visualization method, which is an attribution analysis. We find that the U-RISC requires a larger area around a pixel to predict whether the pixel belongs to the cell membrane or not. Finally, we integrate the currently available methods to provide a new benchmark (0.67, 10% higher than the leader of the competition, 0.61) for cell membrane segmentation on the U-RISC and propose some suggestions in developing deep learning algorithms. The U-RISC dataset and the deep learning codes used in this study are publicly available.

PS-Net: human perception-guided segmentation network for EM cell membrane

“…However, as EM imaging techniques have advanced, the demand for the segmentation of ultra-high-resolution images has increased. For instance, the recently proposed U-RISC dataset ( Shi et al 2022 ) has a resolution of . When applied to this dataset, the performance of these methods significantly decreased (from 98% on ISBI 2012 to 60% on U-RISC).…”

“…Despite the significant progress made by deep-learning (DL) methods ( Ronneberger et al 2015 , Paszke et al 2016 , Chaurasia and Culurciello 2017 , Shen et al 2017 , Yu et al 2017 , Hu et al 2018 , Khadangi et al 2021 ) in the segmentation of EM cell membrane segmentation [ISBI 2012 ( Arganda-Carreras et al 2015 )] approaching or even surpassing human performance, their performance has been observed to deteriorate on high-resolution datasets (both in terms of the absolute number of pixels and the size of each pixel). Specifically, DL methods have achieved ∼98% accuracy (V-Rand) on the ISBI 2012 dataset, but only about 60% on the high-resolution U-RISC dataset ( Shi et al 2022 ) with pixels. In contrast, human performance on both datasets has been found to be consistently high (close to 99%).…”