Interactive Volume Exploration of Petascale Microscopy Data Streams Using a Visualization-Driven Virtual Memory Approach

Abstract

Fig. 1.Our system is the first to enable neuroscientists to interactively explore petascale volume data resulting from high-throughput electron microscopy data streams. The volume can be visualized while the acquisition is still in progress, and without pre-processing all data into a 3D multi-resolution hierarchy as required by all previous systems. Shown here: 21, 494 ⇥ 25, 790 ⇥ 1, 850 mouse cortex.Abstract-This paper presents the first volume visualization system that scales to petascale volumes imaged as a continuous stream of high-resolution electron microscopy images. Our architecture scales to dense, anisotropic petascale volumes because it: (1) decouples construction of the 3D multi-resolution representation required for visualization from data acquisition, and (2) decouples sample access time during ray-casting from the size of the multi-resolution hierarchy. Our system is designed around a scalable multi-resolution virtual memory architecture that handles missing data naturally, does not pre-compute any 3D multi-resolution representation such as an octree, and can accept a constant stream of 2D image tiles from the microscopes. A novelty of our system design is that it is visualization-driven: we restrict most computations to the visible volume data. Leveraging the virtual memory architecture, missing data are detected during volume ray-casting as cache misses, which are propagated backwards for on-demand out-of-core processing. 3D blocks of volume data are only constructed from 2D microscope image tiles when they have actually been accessed during ray-casting. We extensively evaluate our system design choices with respect to scalability and performance, compare to previous best-of-breed systems, and illustrate the effectiveness of our system for real microscopy data from neuroscience.

Real-time 3D reconstruction at scale using voxel hashing

“…This is a well studied topic in the volume rendering literature, with efficient methods based on sparse voxel octrees [Laine and Karras 2011;Kämpe et al 2013], simpler multi-level hierarchies and adaptive data structures [Kraus and Ertl 2002;Lefebvre et al 2005;Bastos and Celes 2008;Reichl et al 2012] and out-of-core streaming architectures for large datasets [Hadwiger et al 2012;Crassin et al 2009]. These approaches have begun to be explored in the context of online reconstruction, where the need to support real-time updates of the underlying data adds a fundamentally new challenge.…”

JiTTree: A Just-in-Time Compiled Sparse GPU Volume Data Structure

“…The octree is traversed with a threshold for sampling the brick data which is used in slice based rasterization. Virtual memory management [10] can be used to complement our brick-based data structure as well, but it is an orthogonal concept to compact sparse representations.…”

Combined Volume Registration and Visualization

“…Accurate data registration is a prerequisite for these systems, in contrast to ours. In [HBJP12] the authors report a system using GPU ray casting to visualize petascale electron microscopy volume data. The authors acknowledge the need for volume registration and stitching and treat it as an external process.…”