A theoretical approach to the use of cyberinfrastructure in geographical analysis

Wang, Shaowen; Armstrong, Marc P.

doi:10.1080/13658810801918509

Cited by 105 publications

(60 citation statements)

References 46 publications

(48 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally, it involves two steps to parallelize a spatial algorithm: to decompose the data (spatial decomposition) and to schedule the tasks (task scheduling) [20][21][22].…”

Section: Methodsmentioning

confidence: 99%

A Stream Tilling Approach to Surface Area Estimation for Large Scale Spatial Data in a Shared Memory System

et al. 2017

View full text Add to dashboard Cite

Surface area estimation is a widely used tool for resource evaluation in the physical world. When processing large scale spatial data, the input/output (I/O) can easily become the bottleneck in parallelizing the algorithm due to the limited physical memory resources and the very slow disk transfer rate. In this paper, we proposed a stream tilling approach to surface area estimation that first decomposed a spatial data set into tiles with topological expansions. With these tiles, the one-to-one mapping relationship between the input and the computing process was broken. Then, we realized a streaming framework towards the scheduling of the I/O processes and computing units. Herein, each computing unit encapsulated a same copy of the estimation algorithm, and multiple asynchronous computing units could work individually in parallel. Finally, the performed experiment demonstrated that our stream tilling estimation can efficiently alleviate the heavy pressures from the I/O-bound work, and the measured speedup after being optimized have greatly outperformed the directly parallel versions in shared memory systems with multi-core processors.

show abstract

“…Generally, it involves two steps to parallelize a spatial algorithm: to decompose the data (spatial decomposition) and to schedule the tasks (task scheduling) [20][21][22].…”

Section: Methodsmentioning

confidence: 99%

A Stream Tilling Approach to Surface Area Estimation for Large Scale Spatial Data in a Shared Memory System

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Typical efforts can be demonstrated in spatial interpolation [26][27][28], spatial statistics [26,[29][30][31], spatial visualization [32], viewshed analysis [33,34], spatial simulation [35,36], geocollaboration [27,37], etc. In most cases, the optimization strategy used in one algorithm may not work for another.…”

Section: Computationally Intensive Geocomputationmentioning

confidence: 99%

“…These fundamental solutions can be the theoretical approaches, such as the grid-based representation of a spatial computational domain [27,30,38], the framework for multilayered libraries between the computing resources and the serial geographic algorithms [39], and the research on the distributed geographic information processing (DGIP) which provides a guiding methodology and principles for implementing geospatial middleware [40][41][42], or the general libraries for processing a particular category of applications, such as the pRPL (parallel raster processing programming library) [43], the PaRGO (parallel raster-based geocomputation operators) [44], etc. In summary, the implementations of these fundamental solutions are mostly based on the powerful computing technologies, including grid computing, cloud computing, and parallel computing.…”

Section: Computationally Intensive Geocomputationmentioning

confidence: 99%

Graph-Based Divide and Conquer Method for Parallelizing Spatial Operations on Vector Data

Kang

Lin

2014

Remote Sensing

View full text Add to dashboard Cite

Abstract:In computer science, dependence analysis determines whether or not it is safe to parallelize statements in programs. In dealing with the data-intensive and computationally intensive spatial operations in processing massive volumes of geometric features, this dependence can be well utilized for exploiting the parallelism. In this paper, we propose a graph-based divide and conquer method for parallelizing spatial operations (GDCMPSO) on vector data. It can represent spatial data dependences in spatial operations through representing the vector features as graph vertices, and their computational dependences as graph edges. By this way, spatial operations can be parallelized in three steps: partitioning the graph into graph components with inter-component edges firstly, simultaneously processing multiple subtasks indicated by the graph components secondly and finally handling remainder tasks denoted by the inter-component edges. To demonstrate how it works, buffer operation and intersection operation under this paradigm are conducted. In a 12-core environment, the two spatial operations both gain obvious performance improvements, and the speedups are more than eight. The testing results suggest that GDCMPSO contributes to a method for parallelizing spatial operations and can greatly improve the computing efficiency on multi-core architectures.

show abstract

“…In addition, the predictive scaling algorithm sheds lights on automatically scaling cloud computing resources for other data processing platforms beyond Hadoop. Finally, we believe that the proposed approach offers a valuable reference in planning better performed spatial applications in a cyberinfrastructure environment to more cost-efficiently address data-and computational intensity challenges in geospatial domains [40,41].…”

mentioning

confidence: 99%

Automatic Scaling Hadoop in the Cloud for Efficient Process of Big Geospatial Data

Yang

Liu

et al. 2016

IJGI

View full text Add to dashboard Cite

Efficient processing of big geospatial data is crucial for tackling global and regional challenges such as climate change and natural disasters, but it is challenging not only due to the massive data volume but also due to the intrinsic complexity and high dimensions of the geospatial datasets. While traditional computing infrastructure does not scale well with the rapidly increasing data volume, Hadoop has attracted increasing attention in geoscience communities for handling big geospatial data. Recently, many studies were carried out to investigate adopting Hadoop for processing big geospatial data, but how to adjust the computing resources to efficiently handle the dynamic geoprocessing workload was barely explored. To bridge this gap, we propose a novel framework to automatically scale the Hadoop cluster in the cloud environment to allocate the right amount of computing resources based on the dynamic geoprocessing workload. The framework and auto-scaling algorithms are introduced, and a prototype system was developed to demonstrate the feasibility and efficiency of the proposed scaling mechanism using Digital Elevation Model (DEM) interpolation as an example. Experimental results show that this auto-scaling framework could (1) significantly reduce the computing resource utilization (by 80% in our example) while delivering similar performance as a full-powered cluster; and (2) effectively handle the spike processing workload by automatically increasing the computing resources to ensure the processing is finished within an acceptable time. Such an auto-scaling approach provides a valuable reference to optimize the performance of geospatial applications to address data-and computational-intensity challenges in GIScience in a more cost-efficient manner.

show abstract

A theoretical approach to the use of cyberinfrastructure in geographical analysis

Cited by 105 publications

References 46 publications

A Stream Tilling Approach to Surface Area Estimation for Large Scale Spatial Data in a Shared Memory System

A Stream Tilling Approach to Surface Area Estimation for Large Scale Spatial Data in a Shared Memory System

Graph-Based Divide and Conquer Method for Parallelizing Spatial Operations on Vector Data

Automatic Scaling Hadoop in the Cloud for Efficient Process of Big Geospatial Data

Contact Info

Product

Resources

About