A Parallel N-Dimensional Space-Filling Curve Library and Its Application in Massive Point Cloud Management

Abstract

Because of their locality preservation properties, Space-Filling Curves (SFC) have been widely used in massive point dataset management. However, the completeness, universality, and scalability of current SFC implementations are still not well resolved. To address this problem, a generic n-dimensional (nD) SFC library is proposed and validated in massive multiscale nD points management. The library supports two well-known types of SFCs (Morton and Hilbert) with an object-oriented design, and provides common interfaces for encoding, decoding, and nD box query. Parallel implementation permits effective exploitation of underlying multicore resources. During massive point cloud management, all xyz points are attached an additional random level of detail (LOD) value l. A unique 4D SFC key is generated from each xyzl with this library, and then only the keys are stored as flat records in an Oracle Index Organized Table (IOT). The key-only schema benefits both data compression and multiscale clustering. Experiments show that the proposed nD SFC library provides complete functions and robust scalability for massive points management. When loading 23 billion Light Detection and Ranging (LiDAR) points into an Oracle database, the parallel mode takes about 10 h and the loading speed is estimated four times faster than sequential loading. Furthermore, 4D queries using the Hilbert keys take about 1~5 s and scale well with the dataset size.

Geospatial Data Management Research: Progress and Future Directions

“…In the last decade there have been significant advancements in the field of 3D/4D geospatial databases. The parallelization of queries using n-dimensional space-filling curves [34] has been examined [35] and applied to massive point clouds [35][36][37][38]. Furthermore, following the concepts of Langran and Stuart [39], spatio-temporal data models including moving surface and solid geometries have been implemented in object-relational [40][41][42][43] and object-oriented [44,45] geospatial database management systems.…”

“…For an efficient management of massive point cloud data, space-filling curves [34] are frequently used on top of object-relational databases [35,36]. Massive data sets were tested within different environments and point cloud benchmarks were developed [37].…”

A Simple Semantic-Based Data Storage Layout for Querying Point Clouds

“…In such instances, scalability quickly becomes an issue [23]. Because file-based approaches are generally based on a proprietary-file format, data sharing among different applications becomes harder [24]. Finally, for the user to run ad-hoc queries, a file-based application is simply not adequate [11,25].…”

MAP-Vis: A Distributed Spatio-Temporal Big Data Visualization Framework Based on a Multi-Dimensional Aggregation Pyramid Model

“…When the server receives the query request, the middleware parses the request parameters into the space/ time/attribute filters. (2) For the spatial filters, the query geometry will be recursively divided into multi-level SFC cells, while the translated spatial ranges are calculated using our recursive approximation algorithm, published in [35]; (3) The temporal and attribute filters will be discretized and appended to the translated spatial ranges and derive the final scan ranges according to the rowkey schema; (4) The scan ranges are sent to HBase for scan operations, filtering out of false positive data from the initial scan results and aggregating final query results for client rendering. In order to avoid excessive I/O communication between servers and clients, the mechanism of the HBase CoProcessor is used to implement the above-mentioned multidimensional query algorithm, as illustrated in Figure 9.…”