Interactive Visualization and On-Demand Processing of Large Volume Data: A Fully GPU-Based Out-of-Core Approach

Abstract

In a wide range of scientific fields, 3D datasets production capabilities have widely evolved in recent years, especially with the rapid increase in their sizes. As a result, many large-scale applications, including visualization or processing, have become challenging to address. A solution to this issue lies in providing out-of-core algorithms specifically designed to handle datasets significantly larger than memory. In this article, we present a new approach that extends the broad interactive addressing principles already established in the field of out-of-core volume rendering on GPUs to allow on-demand processing during the visualization stage. We propose a pipeline designed to manage data as regular 3D grids regardless of the underlying application. It relies on a caching approach with a virtual memory addressing system coupled to an efficient parallel management on GPU to provide efficient access to data in interactive time. It allows any visualization or processing application to leverage the flexibility of its structure by managing multi-modality datasets. Furthermore, we show that our system delivers good performance on a single standard PC with low memory budget on the GPU.

Distributed Out-of-Core Approach for In-Situ Volume Rendering of Massive Dataset

“…It is based on a virtual addressing system using a multi-level multi-resolution page table hierarchy and a brick cache on GPU. The underlying structure, implemented in GPU texture memory, is fully maintained on the GPU to optimize the communication load with the CPU [18]. This system is able to interactively address a whole multi-resolution volume decomposed into small voxel bricks entirely stored on disk with compression.…”

“…A dedicated CPU thread manages the lists of the required bricks to provide them to the GPU after reading and decompressing. Unlike [7] and [18], we are interested here in the distribution of this out-of-core management solution on a heterogeneous node that includes several GPUs and CPUs with an approach including remote rendering allowing a complete in-situ visualization pipeline.…”

“…Compared with these works, we propose to introduce a distribution method of an out-of-core approach based on an efficient GPU virtual addressing and caching system [18] on multi-GPUs, multi-CPUs single node server. Combined with a ray-guided volume renderer with sort-last compositing for interactive visualization on datasets exceeding terabytes.…”

State‐of‐the‐art in Large‐Scale Volume Visualization Beyond Structured Data

“…For a complete review, we refer the reader to the state‐of‐the‐art report by Beyer et al [BHP15]. Recent papers presenting out‐of‐core paging and streaming approaches for structured volume data include the ones by Hadwiger et al [HAAB * 18], by Beyer et al [BMA * 19], by Wang et al [WWJ19], and by Sarton et al [SRL19, SCRL20]. We concentrate specifically on the data management aspects of such systems.…”

MFA-DVR: Direct Volume Rendering of MFA Models

“…In order to overcome memory limitations when rendering large-scale datasets, several methods have been proposed. Out-of-core techniques [17,56] break the data set into chunks that can be rendered independently without using more than a constant amount of memory. Parallel visualization [50][51][52] utilizes distributed computing to visualize large-scale data.…”