Dictionary Compression in Point Cloud Data Management

Abstract

Nowadays, massive amounts of point cloud data can be collected thanks to advances in data acquisition and processing technologies like dense image matching and airborne LiDAR (Light Detection and Ranging) scanning. With the increase in volume and precision, point cloud data offers a useful source of information for natural resource management, urban planning, self-driving cars and more. At the same time, the scale at which point cloud data is produced, introduces management challenges: it is important to achieve efficiency both in terms of querying performance and space requirements. Traditional file-based solutions to point cloud management offer space efficiency, however, cannot scale to such massive data and provide the same declarative power as a database management system (DBMS).In this paper, we propose a time-and space-efficient solution to storing and managing point cloud data in main memory columnstore DBMS. Our solution, Space-Filling Curve Dictionary-Based Compression (SFC-DBC), employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves. It does so by constructing the space-filling curve over a compressed, artificially introduced 3D dictionary space. Consequently, SFC-DBC significantly optimizes query execution, and yet it does not require additional storage resources, compared to traditional dictionary-based compression. With respect to space-filling curve-based approaches, it minimizes storage footprint and increases resilience to skew. As a proof of concept, we develop and evaluate our approach as a research prototype in the context of SAP HANA. SFC-DBC outperforms other dictionary-based compression schemes by up to 61% in terms of space and up to 9.4x in terms of query performance.

Tensor Tucker Decomposition based Geometry Compression on Three Dimensional LiDAR Point Cloud Image

“…As the Morton is too discrete, an optimization algorithm for the Morton is proposed by IT on this basis finally [3]. Mirjana et al, introduced space-filling curve dictionary-based compression, employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves [4]. Javier et al, used a fast variant of Gaussian Mixture Models and an expectation-Maximization algorithm to replace the points grouped in the previous step with a set of Gaussian distributions.…”

Efficient Encoding and Decoding Extended Geocodes for Massive Point Cloud Data

“…However, its encoding/decoding performance is lower than the constant-type code. In [17], Pavlovic proposed a compression method for point cloud data using the SFC. The method is based on a static space and a distribution of coordinates for input point cloud data.…”

Effect of d-Dimensional Re-orderings on Lossless Compression of Radio-Astronomy and Digital Elevation Data

“…They have also been used to reduce I/O access times by reducing the number of disk seeks necessary for range queries [1] or improving disk address prediction from disk indices [4]. SFCs have shown promise in large point-data management software, for both data compression and access [5], [6]. Given their locality preservation properties, SFCs are effective at improving k nearest-neighbour queries [7], [8], [9], [10].…”