GIS navigation boosted by column stores

Alvanaki, Foteini; Gonçalves, Romulo; Ivanova, Milena; Kersten, Martin; Kyzirakos, Kostis

doi:10.14778/2824032.2824110

Cited by 11 publications

(17 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SQL as declarative language is ideal to express complex adhoc queries and data flows. The connection with QGIS [9] for efficient visualization of spatial data is under development [14].…”

Section: Discussionmentioning

confidence: 99%

A Round Table for Multi-disciplinary Research on Geospatial and Climate Data

Gonçalves

Ivanova²,

Alvanaki

et al. 2015

2015 IEEE 11th International Conference on E-Science

Self Cite

View full text Add to dashboard Cite

Abstract-Earth observation sciences produce large sets of data which are inherently rich in spatial and geo-spatial information. Together with live data collected from monitoring systems and large collections of semantically rich objects they provide new opportunities for advanced eScience research on climatology, urban planning and smart cities to name a few.Such combination of heterogeneous data sets forms a new source of knowledge. Efficient knowledge extraction from them is an eScience challenge. It requires efficient bulk data injection from both static and streaming data sources, dynamic adaptation of the physical and logical schema, efficient methods to correlate spatial and temporal data, and flexibility to (re-)formulate the research question at any time.In this work, we present a data management layer over a column-oriented relational data management system that provides efficient analysis of spatiotemporal data. It provides fast data ingestion through different data loaders, tabular and arraybased storage, and a dynamic step-wise exploration.

show abstract

“…SQL as declarative language is ideal to express complex adhoc queries and data flows. The connection with QGIS [9] for efficient visualization of spatial data is under development [14].…”

Section: Discussionmentioning

confidence: 99%

A Round Table for Multi-disciplinary Research on Geospatial and Climate Data

Gonçalves

Ivanova²,

Alvanaki

et al. 2015

2015 IEEE 11th International Conference on E-Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent work [1,12,29] illustrates the potential of column-store DBMSs to meet point cloud management requirements. The MonetDB demo [1] showcases the declarative power of DBMS when managing point cloud data that is enriched with semantics from different sources.…”

Section: Related Workmentioning

confidence: 99%

“…The MonetDB demo [1] showcases the declarative power of DBMS when managing point cloud data that is enriched with semantics from different sources. The solution proposed in [12] focuses on implementation details of existing algorithms for spatial selections and joins on modern hardware, and does not address space efficiency.…”

Section: Related Workmentioning

confidence: 99%

Dictionary Compression in Point Cloud Data Management

Pavlović

Bastian

Gildhoff

et al. 2017

Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

View full text Add to dashboard Cite

Nowadays, massive amounts of point cloud data can be collected thanks to advances in data acquisition and processing technologies like dense image matching and airborne LiDAR (Light Detection and Ranging) scanning. With the increase in volume and precision, point cloud data offers a useful source of information for natural resource management, urban planning, self-driving cars and more. At the same time, the scale at which point cloud data is produced, introduces management challenges: it is important to achieve efficiency both in terms of querying performance and space requirements. Traditional file-based solutions to point cloud management offer space efficiency, however, cannot scale to such massive data and provide the same declarative power as a database management system (DBMS).In this paper, we propose a time-and space-efficient solution to storing and managing point cloud data in main memory columnstore DBMS. Our solution, Space-Filling Curve Dictionary-Based Compression (SFC-DBC), employs dictionary-based compression in the spatial data management domain and enhances it with indexing capabilities by using space-filling curves. It does so by constructing the space-filling curve over a compressed, artificially introduced 3D dictionary space. Consequently, SFC-DBC significantly optimizes query execution, and yet it does not require additional storage resources, compared to traditional dictionary-based compression. With respect to space-filling curve-based approaches, it minimizes storage footprint and increases resilience to skew. As a proof of concept, we develop and evaluate our approach as a research prototype in the context of SAP HANA. SFC-DBC outperforms other dictionary-based compression schemes by up to 61% in terms of space and up to 9.4x in terms of query performance.

show abstract

“…Currently through the works [8], [6], [12], [13], MonetDB spatial features have been matured to provide core technology components for geo-spatial big data analytics. Atomic spatial types and their operations are becoming part of the relational kernel and not an add-on.…”

Section: A Column-oriented Architecturementioning

confidence: 99%

“…Despite column-oriented architectures emerge as the right candidate and the efforts to extend them for spatiotemporal analysis over large data sets [8], [6], [12], [13], their flat storage model is not yet suitable to store a large 3D city model. To do so, we extended a column-store to also support a nested column-oriented storage for 3D city models.…”

Section: Introductionmentioning

confidence: 99%

A columnar architecture for modern risk management systems

Gonçalves

Zlatanova

Kyzirakos

et al. 2016

2016 IEEE 12th International Conference on E-Science (E-Science)

Self Cite

View full text Add to dashboard Cite

Abstract-3D digital city models form the basis for flow simulations (e.g. wind flow and water runoff), urban planning, underand over-ground formation analysis, and they are very important for automated anomaly detection on man made structures. They consist of large collections of semantically rich objects which have many properties such as material and color. Such user's data structure perception is leading to complex storage schemas. The number of table relations to manage and the large data storage footprint drawbacks are then extended with the fact that not all the systems have a "real" 3D data type.In this work we would like to show our efforts to develop a new kind of Spatial Data Management System (SDBMS) where topological and geometric functionality for 3D raster manipulation will become part of the relational kernel and not an add-on. With it spatial analysis tailored to different use case scenarios is done on-demand and fast enough to support real-time interaction in modern risk management systems. I. INTRODUCTIONDigital 3D city models play a crucial role in research of urban phenomena; they form the basis for flow simulations (e.g. wind streams and water runoff), analysis of underground formations and man made structures which provide crucial information for effective risk management systems.An urban scene, represented as a 3D city model, consists of large collections of semantically rich objects which have a number of properties such as use, function, and year. They are commonly reconstructed by segmenting and triangulating a point cloud thereby creating a surface representation. Representing urban objects (e.g. buildings, roads, trees, etc.) as surfaces has drawbacks while calculating intersections and volumes, and creating cross-sections is complex. Furthermore, modeling volumetric objects, such as walls, water, and underground, requires the deployment of complex shapes [18].Such users data structure perception is leading to complex storage schemes. The storage scheme designed for systems like Oracle Spatial, Grass, and PostGIS has limitations such as the management of many tables when the selection predicate is on the 3D city model semantics. The number of table relations to manage and the large data storage footprint drawbacks are then extended with the fact not all the systems have a "real" 3D data type. PostGIS, highly adopted in eScience projects, is a clear example.

show abstract

GIS navigation boosted by column stores

Cited by 11 publications

References 5 publications

A Round Table for Multi-disciplinary Research on Geospatial and Climate Data

A Round Table for Multi-disciplinary Research on Geospatial and Climate Data

Dictionary Compression in Point Cloud Data Management

A columnar architecture for modern risk management systems

Contact Info

Product

Resources

About