A High Performance, Spatiotemporal Statistical Analysis System Based on a Spatiotemporal Cloud Platform

Abstract

Abstract:With the increase in size and complexity of spatiotemporal data, traditional methods for performing statistical analysis are insufficient for meeting real-time requirements for mining information from Big Data, due to both data-and computing-intensive factors. To solve the Big Data challenges in geostatistics and to support decision-making, a high performance, spatiotemporal statistical analysis system (Geostatistics-Hadoop) is proposed in this paper. The proposed system has several features: (1) Hadoop is enhanced to handle spatial data in a native format and execute a number of parallelized spatial analysis algorithms to solve practical geospatial analysis problems; (2) the Oozie-based workflow system is utilized to ease the operation and sharing of spatial analysis services; and (3) a private cloud platform based on Eucalyptus is leveraged to provide on-the-fly and elastic computing resources. Experimental results show that Geostatistics-Hadoop efficiently conducts rapid information mining and analysis of big spatiotemporal data sets, with the support of elastic computing resources from a cloud platform. The adoption of cloud computing and the Hadoop cluster to parallelize statistical calculations significantly improves the performance of Big Data analyses.

Big Earth data analytics: a survey

“…Some choose the private cloud as it allows for full control (Doelitzscher, Sulistio, Reich, Kuijs, & Wolf, 2011), but most adopt the public cloud where a third-party cloud provider performs the updates and maintenance of computing resources (Varia & Mathew, 2014). For example, Mapbox uses Landsat on Amazon Web Services to power Landsat-live, a browser-based map that is constantly refreshed with the latest imagery from the Landsat 8 satellite (Yang, Yu, Hu, Jiang, & Li, 2017a use a hybrid cloud, a combination of these two paradigms that inherits the advantages of both to put the sensitive data/systems in a private cloud while supplying a service to the public cloud for public service (Jin et al, 2017).…”

A Smart Web-Based Geospatial Data Discovery System with Oceanographic Data as an Example

“…The database and models are updated monthly. The technologies used to implement the proposed system for PO.DAAC's dataset are: HDFS, Map/Reduce jobs, Spark, Elasticsearch, and DC2 [30,31]. The experiment was conducted on the NASA AIST cloud platform, a hybrid cloud computing environment provided for scientific research.…”