Efficient spatial query processing for big data

Lee, Kisung; Ganti, Raghu; Srivatsa, Mudhakar; Liu, Ling

doi:10.1145/2666310.2666481

Cited by 51 publications

(26 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spatial support has also been extended to NoSQLbased solutions. [7], [8] adopt Geohash to handle spatial data in HBase. HGrid [23] builds a hybrid index structure, combining a quad-tree and a grid as primary and secondary indices in HBase.…”

Section: Related Workmentioning

confidence: 99%

“…However, they do not have native support for geographic data and spatial queries, which is required in a database of geographic data. Some methods [7], [8] use Geohash * * , a linearization technique that transforms two-dimensional spatial data into a one-dimensional hashcode, to handle spatial data in HBase. However, there are two major limitations of Geohash which they do not address.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

G-HBase: A High Performance Geographical Database Based on HBase

Takasu

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYWith the recent explosion of geographic data generated by smartphones, sensors, and satellites, a data storage that can handle the massive volume of data and support high-computational spatial queries is becoming essential. Although key-value stores efficiently handle large-scale data, they are not equipped with effective functions for supporting geographic data. To solve this problem, in this paper, we present G-HBase, a high-performance geographical database based on HBase, a standard keyvalue store. To index geographic data, we first use Geohash as the rowkey in HBase. Then, we present a novel partitioning method, namely binary Geohash rectangle partitioning, to support spatial queries. Our extensive experiments on real datasets have demonstrated an improved performance with k nearest neighbors and range query in G-HBase when compared with SpatialHadoop, a state-of-the-art framework with native support for spatial data. We also observed that performance of spatial join in G-HBase is on par with SpatialHadoop and outperforms SJMR algorithm in HBase.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

G-HBase: A High Performance Geographical Database Based on HBase

Takasu

2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Since the practical method to efficiently query against big spatial data is to employ the divide and conquer strategy [9,26], most MapReduce-based PSQPAs use certain types of space filling curves, such as Hilbert space-filling curve, to map MBRs to grids based on the spatial correlation for optimizing efficiency [27,28]. We simply treat the number of grids p as one of the internal parameters of Spark-based PSQPAs.…”

Section: Identifying Factors Impacting the Efficiency Of Spark-based mentioning

confidence: 99%

Elastic Spatial Query Processing in OpenStack Cloud Computing Environment for Time-Constraint Data Analysis

Huang

Zhang

et al. 2017

IJGI

View full text Add to dashboard Cite

Abstract:Geospatial big data analysis (GBDA) is extremely significant for time-constraint applications such as disaster response. However, the time-constraint analysis is not yet a trivial task in the cloud computing environment. Spatial query processing (SQP) is typical computation-intensive and indispensable for GBDA, and the spatial range query, join query, and the nearest neighbor query algorithms are not scalable without using MapReduce-liked frameworks. Parallel SQP algorithms (PSQPAs) are trapped in screw-processing, which is a known issue in Geoscience. To satisfy time-constrained GBDA, we propose an elastic SQP approach in this paper. First, Spark is used to implement PSQPAs. Second, Kubernetes-managed Core Operation System (CoreOS) clusters provide self-healing Docker containers for running Spark clusters in the cloud. Spark-based PSQPAs are submitted to Docker containers, where Spark master instances reside. Finally, the horizontal pod auto-scaler (HPA) would scale-out and scale-in Docker containers for supporting on-demand computing resources. Combined with an auto-scaling group of virtual instances, HPA helps to find each of the five nearest neighbors for 46,139,532 query objects from 834,158 spatial data objects in less than 300 s. The experiments conducted on an OpenStack cloud demonstrate that auto-scaling containers can satisfy time-constraint GBDA in clouds.

show abstract

“…But, most of them require internal modifications of frameworks. Lee et al (2014) have come up with a spatial index especially for big data, based on a hierarchical spatial data structure stored in distributed file storage systems. This spatial index has several advantages; such as: it can be implemented without changing the internal implementation of the existing storage systems, simple and efficient filtering, and supports to updates of spatial objects.…”

Section: The Algorithm Had Following Three Phases (1) Computation Of mentioning

confidence: 99%

Geo-spatial Big Data Mining Techniques

Alkathiri¹,

Abdul²,

Potdar³

2016

IJCA

View full text Add to dashboard Cite

As stated in literature by several authors, there has been literally big-bang explosion in data acquired in recent times. This is especially so about the geographical or geospatial data. The huge volume of data acquired in different formats, structured, unstructured ways, having large complexity and non-stop generation of these data have posed an insurmountable challenge in scientific and business world alike. The conventional tools, techniques and hardware existing about a decade ago have met with the limitations in handling such data. Hence, such data are termed as big data. This has necessitated inventing new software tools and techniques as well as parallel computing hardware architectures to meet the requirement of timely and efficient handling of the big data. The field of data mining has been benefitted from these evolutions as well. This article reviews the evolution of data mining techniques over last two decades and efforts made in developing big data analytics, especially as applied to geospatial big data. This is still a very actively evolving field. There will be no surprise if some new techniques are published before this article appears in print.

show abstract

Efficient spatial query processing for big data

Cited by 51 publications

References 11 publications

G-HBase: A High Performance Geographical Database Based on HBase

G-HBase: A High Performance Geographical Database Based on HBase

Elastic Spatial Query Processing in OpenStack Cloud Computing Environment for Time-Constraint Data Analysis

Geo-spatial Big Data Mining Techniques

Contact Info

Product

Resources

About