Automatic Neural Reconstruction from Petavoxel of Electron Microscopy Data

Abstract

Connectomics is the study of the dense structure of the neurons in the brain and their synapses, providing new insights into the relation between brain's structure and its function. Recent advances in Electron Microscopy enable high-resolution imaging (4nm per pixel) of neural tissue at a rate of roughly 10 terapixels in a single day, allowing neuroscientists to capture large blocks of neural tissue in a reasonable amount of time. The large amounts of data require novel computer vision based algorithms and scalable software frameworks to process this data. We describe RhoANA [1], our dense Automatic Neural Annotation framework, which we have developed in order to automatically align, segment and reconstruct a 1mm 3 brain tissue (~2 peta-pixels).Our workflow is depicted in Fig. 1. A Zeiss MultiSEM 505 (Fig 1-a) captures the images (Fig 1-b) of a neural tissue block that is sliced into thin, 30nm thick, sections using an automatic microtome. The images of each section are stitched together and the entire volume is 3D aligned using our stitching and registration tool (Fig 1-c). Each cell membrane is detected using a membrane classification algorithm, and the cells are reconstructed into 3D objects (Fig 1-d, 1-e). Finally, we use our web-based tool to manually proofread the output, and ensure reconstruction correctness (Fig 1-f). The rest of the paper details the software tools that are used after the image acquisition by the microscope.The Zeiss MultiSEM 505 simultaneously captures 61 image tiles (multi-beam field-of-view, or mFoV) at a time in a hexagonal shape (Fig 2). Multiple mFoVs are captured to cover the targeted region in each section (Fig 2). To give immediate feedback, we have developed a web-based application that allows viewing a section after it is acquired by the microscope. Our application can also overlay additional information on each mFoV (e.g., image quality score), and perform basic image processing operations.The first stage of the pipeline performs 2D stitching and 3D registration. When collecting multiple mFoVs at the nanometer resolution, the microscope estimation of image location is imprecise. The stitching process extracts and matches features from the boundaries of each pair of adjacent tiles, and finds a per-tile rigid transformation that minimizes the distances of these features. 3D registration finds a per-section elastic (non-affine) transformation that compensates for non-linear distortions, and is based on TrakEM2's alignment tool [2]. The registration overlays a grid of points on each section that need to be matched to the adjacent section. A small image patch is taken around each of the points, and a corresponding patch is searched for in a restricted area of the adjacent section. Finally, an optimization process minimizes the deformation by simulating an elastic system of springs connected between the grid points, and outputs a per-tile transformation that can be used to render the tile. To overcome artifacts that deform parts of the sections (e.g., dirt) and make the alignment ...

Scalable Interactive Visualization for Connectomics

“…Fast and scalable 2D visualizations enable quick signal-to-noise ratio and contrast assessments across image tiles during acquisition. Rapid progress in automatic sample preparation and EM acquisition techniques make it possible to generate a 1 mm 3 volume of brain tissue in less than six months, with each voxel of size 4 × 4 × 30 nm 3 resulting in 2 petabytes of image data [6,7].…”

“…Then, a stack of sections must be aligned into a 3D scan. Here, visualizing stitched tiles and sections allows quick human assessment of alignment quality in addition to computing quantitative measures [6].…”

“…We demonstrate this with the Zeiss MultiSEM 505 electron microscope with which we capture our imagery. This microscope simultaneously captures 61 image tiles, each of roughly 3000 × 2700 pixels, to achieve acquisition rates of approximately 1 terapixel per hour [6,7]. Each simultaneous capture is called a multi-beam field-of-view, or MFOV.…”

“…The Zeiss MultiSEM 505 electron microscope can generate roughly 1 terapixel per hour split across many 61-tile MFOVs and sections [6]. Neuroscientists must look at each tile to assess whether acquisition was of sufficient quality, otherwise the relevant area of the physical sample needs to be located and tagged for re-scanning.…”

High precision automated detection of labeled nuclei in Gigapixel resolution image data of Mouse Brain

“…These segmentations were then combined into geometrically consistent 3D objects by segmentation fusion. A more recent work in automatic neural reconstruction from petavoxel of Electron Microscopy data [53] suggested a dense Automatic Neural Annotation framework (RhoANA) to automatically align, segment and reconstruct a 1mm 3 brain tissue (∼ 2 peta-pixels), using a web-based tool to manually proofread the output, and ensure reconstruction correctness. The pipeline performs membrane classification and 2D segmentation using state-of-the-art deep learning (DL) techniques in order to generate membrane probability maps.…”