Architecting a Query Compiler for Spatial Workloads

Abstract

Modern location-based applications rely extensively on the ecient processing of spatial data and queries. Spatial query engines are commonly engineered as an extension to a relational database or a cluster-computing framework. Large parts of the spatial processing runtime is spent on evaluating spatial predicates and traversing spatial indexing structures. Typical high-level implementations of these spatial structures incur signicant interpretive overhead, which increases latency and lowers throughput. A promising idea to improve the performance of spatial workloads is to leverage native code generation techniques that have become popular in relational query engines. However, architecting a spatial query compiler is challenging since spatial processing has fundamentally dierent execution characteristics from relational workloads in terms of data dimensionality, indexing structures, and predicate evaluation.In this paper, we discuss the underlying reasons why standard query compilation techniques are not fully eective when applied to spatial workloads, and we demonstrate how a particular style of query compilation based on techniques borrowed from partial evaluation and generative programming manages to avoid most of these diculties by extending the scope of custom code generation into the data structures layer. We extend the LB2 main-memory query compiler, a relational engine developed in this style, with spatial data types, predicates, indexing structures, and operators. We show that the spatial extension matches the performance of specialized library code and outperforms relational and map-reduce extensions.

How Good Are Modern Spatial Libraries?

“…The unprecedented rise of location-based services has led to a considerable amount of research efforts that have been focused on four broad areas; (1) systems that scale out [2-4, 9, 10, 17, 58, 59, 61, 69, 71, 72], (2) support for spatial processing in databases [14,32,35,38,41], (3) improving spatial query processing [12, 22-26, 42, 43, 46, 49, 62-64, 74], and (4) leveraging modern hardware and compiling techniques [6,7,27,[54][55][56]73], to handle the increasing demands of applications today.…”

The Case for Learned Spatial Indexes

“…scale and completed 10 billion rides in 2018 [46]. The unprecedented rate of generation of location data has led to a considerable amount of research efforts that have been focused on, systems that scale out [1,2,8,9,14,39,40,41,48,50,51], databases [12,26,27,31,33], improving spatial query processing [11,18,19,20,35,42,43,44,53,34], or leveraging modern hardware and compiling techniques [6,7,37,36,38,52], to handle the increasing demands of applications today.…”