QoS-Aware Approximate Query Processing for Smart Cities Spatial Data Streams

Abstract

Large amounts of georeferenced data streams arrive daily to stream processing systems. This is attributable to the overabundance of affordable IoT devices. In addition, interested practitioners desire to exploit Internet of Things (IoT) data streams for strategic decision-making purposes. However, mobility data are highly skewed and their arrival rates fluctuate. This nature poses an extra challenge on data stream processing systems, which are required in order to achieve pre-specified latency and accuracy goals. In this paper, we propose ApproxSSPS, which is a system for approximate processing of geo-referenced mobility data, at scale with quality of service guarantees. We focus on stateful aggregations (e.g., means, counts) and top-N queries. ApproxSSPS features a controller that interactively learns the latency statistics and calculates proper sampling rates to meet latency or/and accuracy targets. An overarching trait of ApproxSSPS is its ability to strike a plausible balance between latency and accuracy targets. We evaluate ApproxSSPS on Apache Spark Structured Streaming with real mobility data. We also compared ApproxSSPS against a state-of-the-art online adaptive processing system. Our extensive experiments prove that ApproxSSPS can fulfill latency and accuracy targets with varying sets of parameter configurations and load intensities (i.e., transient peaks in data loads versus slow arriving streams). Moreover, our results show that ApproxSSPS outperforms the baseline counterpart by significant magnitudes. In short, ApproxSSPS is a novel spatial data stream processing system that can deliver real accurate results in a timely manner, by dynamically specifying the limits on data samples.

SpatialSSJP: QoS-Aware Adaptive Approximate Stream-Static Spatial Join Processor

“…Approximate Query Processing (AQP) is a computing field that is concerned with cases where exact solutions are either not necessarily needed or too expensive to put into action [2]. It depends on serving approximate results with rigorous errorbounds expressed, for example, in terms of confidence intervals.…”

“…It depends on serving approximate results with rigorous errorbounds expressed, for example, in terms of confidence intervals. What makes AQP significant is the observation that users normally accept to forego tiny accuracy loss for the sake of a high speedup gain [2,9]. In addition, policy makers typically can make discernibly accurate decisions even while query responses are not perfectly accurate.…”

“…As such, we depend on the theory of stratification [14] for calculating an estimate for an acceptable sampling fraction based on a 'margin of error'. We specifically apply (2). 𝑛 = 𝑧 𝛼/2 2 𝑣/𝑒 𝑑𝑒𝑠 2…”

“…AQP is the most adopted solution for the online information overloading problem [2]. This is so because approximate query responses with rigorous error bounds commonly suffice for strategic decision making in smart city scenarios [2].…”

“…AQP is the most adopted solution for the online information overloading problem [2]. This is so because approximate query responses with rigorous error bounds commonly suffice for strategic decision making in smart city scenarios [2]. For example, generating highly accurate 2 > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) < interactive heatmaps from sensors data depending on a geospatially representative well-balanced sample, which is an operation that depends on stream-static geospatial join.…”