SpatialSSJP: QoS-Aware Adaptive Approximate Stream-Static Spatial Join Processor

Abstract

The widespread adoption of Internet of Things (IoT) motivated the emergence of mixed workload scenarios in smart cities, where fast arriving geo-referenced massive amounts data streams need to be joined with archive tables, at scale. This aims at enriching streams with descriptive attributes that enable deeper insightful analytics. More applications are now relying on finding, in real-time, to which geographical region each data streaming spatially-tagged tuple belongs. This problem requires a computationally intensive stream-static join operation, where one side of join is a dynamic stream while the other is a diskresident static table. Even with emergence of some libraries that solve this problem in static-static fashion, their adoption for live scenarios is challenging because join operations are expensive in real-time. In addition, the time-varying nature of fluctuation and skewness in the geospatial data loads arriving online calls for an approximate solution that can trade-off QoS constraints in a way which ensures that the system survives sudden spikes in data loads. In this paper, we present SpatialSSJP, an adaptive spatialaware approximate query processing system that specifically focuses on stream-static joins in a way that guarantees achieving an agreed set of Quality-of-Service goals and maintains geostatistics of stateful online aggregations over stream-static join results. SpatialSSJP employs a state-of-art stratified-like sampling design to select well-balanced representative geospatial data stream samples and serve them to a stream-static geospatial join operator downstream. We implemented a prototype atop Spark Structured Streaming. Our extensive evaluations on big real datasets show that our system can survive and mitigate harsh join workloads and outperform state-of-art baselines by significant magnitudes, without risking rigorous error bounds in terms of the accuracy of the output results. SpatialSSJP achieves a relative accuracy gain against plain Spark joins of approximately 10% in worst cases but reaching up to 50% in best case scenarios.