Towards Parallel Spatial Query Processing for Big Spatial Data

Zhong, Yue; Han, Jizhong; Zhang, Tieying; Li, Zhenhua; Fang, Jinyun; Chen, Guihai

doi:10.1109/ipdpsw.2012.245

Cited by 57 publications

(27 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the practical method to efficiently query against big spatial data is to employ the divide and conquer strategy [9,26], most MapReduce-based PSQPAs use certain types of space filling curves, such as Hilbert space-filling curve, to map MBRs to grids based on the spatial correlation for optimizing efficiency [27,28]. We simply treat the number of grids p as one of the internal parameters of Spark-based PSQPAs.…”

Section: Identifying Factors Impacting the Efficiency Of Spark-based mentioning

confidence: 99%

“…Since a (Graphics Processing Unit) GPU-based parallel requires significant effort in redesigning relevant algorithms, t state-of-art research directs parallel SQP (PSQP) for handling big spatial data in the cloud computing environment [8,9]. For example, Zhong et al [9] implemented several MapReduce-based spatial query operators for parallel SQP algorithms (PSQPAs). Although MapReduce-based PSQPAs perform well with enhanced scalability, the efficiency of the PSQPAs depends on their Hadoop-based property system.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Elastic Spatial Query Processing in OpenStack Cloud Computing Environment for Time-Constraint Data Analysis

Huang

Zhang

et al. 2017

IJGI

View full text Add to dashboard Cite

Abstract:Geospatial big data analysis (GBDA) is extremely significant for time-constraint applications such as disaster response. However, the time-constraint analysis is not yet a trivial task in the cloud computing environment. Spatial query processing (SQP) is typical computation-intensive and indispensable for GBDA, and the spatial range query, join query, and the nearest neighbor query algorithms are not scalable without using MapReduce-liked frameworks. Parallel SQP algorithms (PSQPAs) are trapped in screw-processing, which is a known issue in Geoscience. To satisfy time-constrained GBDA, we propose an elastic SQP approach in this paper. First, Spark is used to implement PSQPAs. Second, Kubernetes-managed Core Operation System (CoreOS) clusters provide self-healing Docker containers for running Spark clusters in the cloud. Spark-based PSQPAs are submitted to Docker containers, where Spark master instances reside. Finally, the horizontal pod auto-scaler (HPA) would scale-out and scale-in Docker containers for supporting on-demand computing resources. Combined with an auto-scaling group of virtual instances, HPA helps to find each of the five nearest neighbors for 46,139,532 query objects from 834,158 spatial data objects in less than 300 s. The experiments conducted on an OpenStack cloud demonstrate that auto-scaling containers can satisfy time-constraint GBDA in clouds.

show abstract

Section: Identifying Factors Impacting the Efficiency Of Spark-based mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Elastic Spatial Query Processing in OpenStack Cloud Computing Environment for Time-Constraint Data Analysis

Huang

Zhang

et al. 2017

IJGI

View full text Add to dashboard Cite

show abstract

“…Afsin Akdogan et al improved the efficiency of parallel queries by building a distributed Voronoi diagram as a flat spatial index [27]. VegaGiStore implements an "indexing + MapReduce" data processing architecture to provide efficient spatial query processing over SBD and numerous concurrent user queries [28]. In addition, many spatial processing platforms based on Hadoop, such as Hadoop-GIS [29] and SpatialHadoop [30], have been proposed.…”

Section: Spatial Querying On a Distributed Platformmentioning

confidence: 99%

Real-Time Spatial Queries for Moving Objects Using Storm Topology

Zhang

Zheng

et al. 2016

IJGI

View full text Add to dashboard Cite

Abstract:With the rapid development of mobile data acquisition technology, the volume of available spatial data is growing at an increasingly fast pace. The real-time processing of big spatial data has become a research frontier in the field of Geographic Information Systems (GIS). To cope with these highly dynamic data, we aim to reduce the time complexity of data updating by modifying the traditional spatial index. However, existing algorithms and data structures are based on single work nodes, which are incapable of handling the required high numbers and update rates of moving objects. In this paper, we present a distributed spatial index based on Apache Storm, an open-source distributed real-time computation system. Using this approach, we compare the range and K-nearest neighbor (KNN) query efficiency of four spatial indexes on a single dataset and introduce a method of performing spatial joins between two moving datasets. In particular, we build a secondary distributed index for spatial join queries based on the grid-partition index. Finally, a series of experiments are presented to explore the factors that affect the performance of the distributed index and to demonstrate the feasibility of the proposed distributed index based on Storm. As a real-world application, this approach has been integrated into an information system that provides real-time traffic decision support.

show abstract

“…Hence, the grouping of nodes based on a specific criteria to store geographically closer events is not doable. Another scalable scheme proposed in [14] named VegaGiStore using multi-tier approach to store, index and retrieve spatial data within big data environment. The approach is based on MapReduce paradigm to distribute and parallelize the query processing.…”

Section: Related Workmentioning

confidence: 99%