Culling for Extreme-Scale Segmentation Volumes: A Hybrid Deterministic and Probabilistic Approach

Abstract

With the rapid increase in raw volume data sizes, such as terabyte-sized microscopy volumes, the corresponding segmentation label volumes have become extremely large as well. We focus on integer label data, whose efficient representation in memory, as well as fast random data access, pose an even greater challenge than the raw image data. Often, it is crucial to be able to rapidly identify which segments are located where, whether for empty space skipping for fast rendering, or for spatial proximity queries. We refer to this process as culling. In order to enable efficient culling of millions of labeled segments, we present a novel hybrid approach that combines deterministic and probabilistic representations of label data in a data-adaptive hierarchical data structure that we call the label list tree. In each node, we adaptively encode label data using either a probabilistic constant-time access representation for fast conservative culling, or a deterministic logarithmic-time access representation for exact queries. We choose the best data structures for representing the labels of each spatial region while building the label list tree. At run time, we further employ a novel query-adaptive culling strategy. While filtering a query down the tree, we prune it successively, and in each node adaptively select the representation that is best suited for evaluating the pruned query, depending on its size. We show an analysis of the efficiency of our approach with several large data sets from connectomics, including a brain scan with more than 13 million labeled segments, and compare our method to conventional culling approaches. Our approach achieves significant reductions in storage size as well as faster query times.

A Method for 3D Reconstruction and Virtual Reality Analysis of Glial and Neuronal Cells

“…Large-scale segmentation is, to date, mostly limited to connectomics; therefore, computer scientists are most interested in providing frameworks for integrated visualizations of large, annotated datasets and analyze connectivity patterns inferred by the presence of synaptic contacts 17,18 . Nevertheless, accurate 3D reconstructions can be used for quantitative morphometric analyses, rather than qualitative assessments of the 3D structures.…”

Multivariate Probabilistic Range Queries for Scalable Interactive 3D Visualization

“…They can report false positive results, but no false negatives. Bloom filters are widely used for database indexing [11], but have also enabled efficient culling of segmented volume data [9]. There exist many variants, such as counting Bloom filters [46], spectral bloom filters [16], and tree-structured Bloom filters [17,20].…”

Interactive Volumetric Visual Analysis of Glycogen‐derived Energy Absorption in Nanometric Brain Structures

“…Finally, Borboor et al [BJA*19] proposed a workflow for the visualization of neuronal structures in wide‐field microscopy images of brain samples, which uses a gradient‐based distance transform and multi‐scale enhancement filters. Our system comprises a scalable volume rendering component extending the method presented by Hadwiger et al [HBJP12], and integrating an hybrid object‐based empty space skipping scheme [HAAB*18] and a data structure for hybrid culling of integer label data [BMA*19], in order to enable real‐time inspection of volumetric EM data together with label data and additional metadata.…”

“… a visualization‐driven scalable multi‐resolution memory architecture built around 2D microscope image tiles, which decouples sample access time during ray‐casting from the size of the multi‐resolution hierarchy [HBJP12]; a high‐performance hybrid ray casting method that scales to multiple volumes of several teravoxels each, enabling the concurrent visualization and querying of the original EM volume, additional segmentation volumes, and additional meta data comprising a variety of neuronal data attributes [BAAK*13]; a method for performing efficient empty space skipping during real time ray casting, by using an hybrid strategy that balances the computational load between determining empty ray segments in a rasterization (object‐order) stage, and sampling non‐empty volume data in the ray‐casting (image‐order) stage [HAAB*18]; a method for performing efficient culling of objects and spatial queries which combines deterministic and probabilistic representations of label data in a data‐adaptive hierarchical data structure [BMA*19]. …”