Out-of-core visualization using iterator-aware multidimensional prefetching

Abstract

1Visualization of multidimensional data presents special challenges for the design of efficient out-of-core data access. Elements that are nearby in the visualization may not be nearby in the underlying data file, which can severely tax the operating system's disk cache. The Granite Scientific Database System can address these problems because it is aware of the organization of the data on disk, and it knows the visualization method's pattern of access. The access pattern is expressed using a toolkit of iterators that both describe the access pattern and perform the iteration itself. Because our system has knowledge of both the data organization and the access pattern, we are able to provide significant performance improvements while hiding the details of out-of-core access from the visualization programmer. This paper presents a brief description of our disk access system placing special emphasis on the benefits offered to a visualization application. We describe a simple demonstration application that shows dramatic performance improvements when used with the 39GB Visible Woman Dataset.

Seeing through the window: pre-fetching strategies for out-of-core image processing algorithms

“…After presenting the method and giving some application examples in Section 2, we set the test scenario in Section 3. In Section 4, we directly compare the results we obtained with those by Rhodes et al, 15,16 because their algorithm is the most similar to our approach.…”

“…Comparing to the solution proposed by Rhodes et al and Lipsa et al, [15][16][17] their cache is only updated in the event of a cache miss. Even when a separate thread was used, the block requested was synchronously fetched and the next candidate block (according to the iterator) was fetched by the thread.…”

“…13,14 Rhodes et al presented a very good survey on the use of cache and pre-fetching strategies and introduced the concept of an iterator aware cache applied to out-of-core volume visualisation applications, with no need to reorganize the original file. 15,16 They built their cache to match the structure of the medical file and to cope with the way the visualisation algorithm traverses the data. They also mentioned the use of a separate thread of execution that loads the next block of data whenever it is requested.…”

Out-of-Core Wavefront Computations with Reduced Synchronization

“…This kind of technique has been successfully used in [13,14,17]. Then, we relax the synchronization constraints between the processors to improve the communication/computation overlap.…”

Iteration Aware Prefetching for Remote Data Access

“…In [Rhodes05], we demonstrated the effectiveness of IAP for visualization of large volumetric scientific datasets. In this paper, we concentrate on rectilinear iteration, in which the direction of iteration is aligned with the major axes.…”