Accelerated EM Connectome Reconstruction using 3D Visualization and Segmentation Graphs

Abstract

Recent advances in automatic image segmentation and synapse prediction in electron microscopy (EM) datasets of the brain enable more efficient reconstruction of neural connectivity. In these datasets, a single neuron can span thousands of images containing complex tree-like arbors with thousands of synapses. While image segmentation algorithms excel within narrow fields of views, the algorithms sometimes struggle to correctly segment large neurons, which require large context given their size and complexity. Conversely, humans are comparatively good at reasoning with large objects. In this paper, we introduce several semi-automated strategies that combine 3D visualization and machine guidance to accelerate connectome reconstruction. In particular, we introduce a strategy to quickly correct a segmentation through merging and cleaving, or splitting a segment along supervoxel boundaries, with both operations driven by affinity scores in the underlying segmentation. We deploy these algorithms as streamlined workflows in a tool called Neu3 and demonstrate superior performance compared to prior work, thus enabling efficient reconstruction of much larger datasets. The insights into proofreading from our work clarify the trade-offs to consider when tuning the parameters of image segmentation algorithms.

A connectome and analysis of the adult Drosophila central brain

“…Neu3 (Hubbard et al, 2020). The fragments that would result in largest connectivity changes were considered first, exploiting automatic guesses through focused proofreading where possible.…”

“…Segmentation errors can be roughly grouped into two classes - ‘false merges’, in which two separate neurons are mistakenly merged together, and ‘false splits’, in which a single neuron is mistakenly broken into several segments. Enabled by advances in visualization and semi-automated proofreading using our Neu3 tool ( Hubbard et al, 2020 ), we first addressed large false mergers. A human examined each putative neuron and determined if it had an unusual morphology suggesting that a merge might have occurred, a task still much easier for humans than machines.…”

“…Finally, to achieve the highest reconstruction completeness possible in the time allotted, and to enable confidence in weaker neuronal pathways, proofreaders connected remaining isolated fragments (segments) to already constructed neurons, using NeuTu ( Zhao et al, 2018 ) and Neu3 ( Hubbard et al, 2020 ). The fragments that would result in largest connectivity changes were considered first, exploiting automatic guesses through focused proofreading where possible.…”

Neuronal Subcompartment Classification and Merge Error Correction

“…1a) to have very few merge errors via an oversegmentation consensus procedure [5], so we focused on errors in SV agglomeration. Once agglomeration errors are identified and localized, they can be fixed efficiently by simply removing the bad agglomeration graph edges, either under human review [18] or automatically. We used subcompartment predictions to identify all somas and branches, and then to detect and correct two classes of agglomeration errors: axon/dendrite branch merge errors, and soma/neurite merge errors.…”

“…Prior work used these cues to tune agglomeration via sparse subcompartment predictions in a multicut setting, which optimizes over an explicit edge-weighted supervoxel graph [16,17]. Alternatively, violation of biological priors in subcompartment predictions can be used to detect post-agglomeration reconstruction errors, which can then be flagged for efficient human proof-reading workflows [18], or fully automated error correction.…”