Distance histogram computation based on spatiotemporal uniformity in scientific data

Abstract

Large data generated by scientific applications imposes challenges in storage and efficient query processing. Many queries against scientific data are analytical in nature and require super-linear computation time using straightforward methods. Spatial distance histogram (SDH) is one of the basic queries to analyze the molecular simulation (MS) data, and it takes quadratic time to compute using brute-force approach. Often, an SDH query is executed continuously to analyze the simulation system over a period of time. This adds to the total time required to compute SDH. In this paper, we propose an approximate algorithm to compute SDH efficiently over consecutive time periods. In our approach, data is organized into a Quad-tree based data structure. The spatial locality of the particles (at given time) in each node of the tree is acquired to determine the particle distribution. Similarly, the temporal locality of particles (between consecutive time periods) in each node is also acquired. The spatial distribution and temporal locality are utilized to compute the approximate SDH at every time instant. The performance is boosted by storing and updating the spatial distribution information over time. The efficiency and accuracy of the proposed algorithm is supported by mathematical analysis and results of extensive experiments using biological data generated from real MS studies.

Computing Spatial Distance Histograms for Large Scientific Data Sets On-the-Fly

“…This work extends the idea of [10] by providing rigorous analysis, empirical evaluation of the algorithm, and implementation in modern GPUs to boost performance.…”

“…The result is a solution that is both practical and efficient, delivering very accurate results in (almost) real-time manner. A shorter version of this paper can be found in [10]: this paper extends [10] by introducing a rigorous analysis of the key distribution function in Section IV, a GPU version of the algorithm in Section VII, and enhanced performance analysis in Section VI.…”

“…However, our analysis in Section VI shows that such errors are negligible. Our experimental results in [10] show that there are a large number of pairs of cells whose density ratio products are around 1.0, thus providing sufficient savings of computation.…”

“…Use of shared memory for other algorithms would not improve the performance due to following reasons: (1) multiple tree levels can’t be loaded into (limited size) shared memory for A 1; (2) advantages of temporal locality in A 2 and A 4 are shadowed by time required to load into, and access from, shared memory. Also, the temporal locality property in A 2 and A 4 increases histogram erros [10]. …”

“…Preliminary results addressing the problem of computing approximate SDH using spatial and temporal uniformities were first reported in [10] (contributions 1 and 2). This work extends the idea of [10] by providing rigorous analysis, empirical evaluation of the algorithm, and implementation in modern GPUs to boost performance.…”