Analyzing spatial analytics systems based on Hadoop and Spark: A user perspective

Abstract

Summary Spatial analytics systems (SASs) represent a technology capable of managing huge volumes of spatial data using frameworks such as Apache Hadoop and Apache Spark. An increasing number of SASs have been proposed, requiring a comparison among them. However, existing comparisons in the literature provide a system‐centric view based on performance evaluations. Thus, there is a lack of comparisons based on the user‐centric view, that is, comparisons that help users to understand how the characteristics of SASs are useful to meet the specific requirements of their spatial applications. In this article, we provide a user‐centric comparison of the following SASs based on Hadoop and Spark: Hadoop‐GIS, SpatialHadoop, SpatialSpark, GeoSpark, GeoMesa Spark, SIMBA, LocationSpark, STARK, Magellan, SparkGIS, and Elcano. This comparison employs an extensive set of criteria related to the general characteristics of these systems, to the aspects of spatial data handling, and to the aspects inherent to distributed systems. Based on this comparison, we introduce guidelines to help users to choose an appropriate SAS. We also describe two case studies based on real‐world applications to illustrate the use of these guidelines. Finally, we discuss chronological tendencies related to SASs and identify limitations that SASs should address to improve user experience.

Combining Fog and Cloud Computing to Support Spatial Analytics in Smart Cities

“…There are several spatial operations that can explore the relationships between spatial data, as categorised in [Castro et al 2020]. In this article, we are interested in topological predicates, metric…”

“…Because there are several SASs available in the literature with different characteristics and capabilities, choosing the most appropriate SAS can become considerably challenging. Thus, smart city managers should use as a basis of choice the state-of-the-art user-centric comparison of existing SASs described in [Castro et al 2020]. For a system-centric view of SASs, the work of Pandey et al (2018) should be referred, as it compares several SASs in the literature based on their query processing performance.…”

“…According to the proposed architecture (Section 4) and guidelines (Section 5), the case study application should be implemented as follows: (i) use of the Apache Airflow to perform the ETL process; (ii) storage of the SDW data in the HDFS as the application requires data storage in the cloud; and (iii) selection of Sedona [Yu et al 2019] as the SAS to process the spatial queries aimed to decisionmaking, as it complies with the application's requirements regarding performance and spatial queries, as well as supports the SQL programming language through the use of SedonaSQL [Pandey et al 2018;Castro et al 2020].…”

“…To deal with big data, the management of the SDW can benefit from the use of a cloud computing environment as infrastructure and from the employment of a parallel and distributed data processing framework, such as Hadoop [Shvachko et al 2010] and Spark [Zaharia et al 2016], to reduce the complexity of the cloud. The processing of the spatial queries can also benefit from the use of Spatial Analytics Systems (SASs), which are developed on top of those frameworks to provide extended functionalities to deal with spatial data and spatial predicates [Castro et al 2020].…”

Efficient Group K Nearest-Neighbor Spatial Query Processing in Apache Spark

“…Recently, several spatial analytics systems (SASs) were proposed to support different type of spatial queries (e.g., range queries, nearest neighbor queries, and spatial joins) over large-scale spatial datasets on shared-nothing clusters in distributed environments. These SASs are mainly based on Hadoop MapReduce or Spark, and several surveys were recently published to describe and classify them [15][16][17][18]. The most representative Spark-based SASs are SpatialSpark [19], GeoSpark (currently Sedona) [4], Simba [20], LocationSpark [21], STARK [22], SparkGIS [23], Elcano [24] and Beast [25].…”