Optimized Data I/O Strategy of the Algorithm of Parallel Digital Terrain Analysis

Li, Yan; Dou, Wanfeng; Yang, Kun; Miao, Shoushuai

doi:10.1109/dcabes.2014.10

Cited by 6 publications

(2 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many studies on the development of parallel programs that take into account the specifics of algorithms and architecture of parallel computing systems. Examples of such studies are [31,17,25,22,32,18,24]. These studies improve the efficiency of implementing specific algorithms or implementing algorithms on parallel computing systems of a particular architecture.…”

Section: Introductionmentioning

confidence: 99%

Parallelism Resource of Numerical Algorithms. Version 1

Aleeva¹,

Aleev²

2022

Preprint

View full text Add to dashboard Cite

The paper is devoted to an approach to solving a problem of the efficiency of parallel computing. The theoretical basis of this approach is the concept of a Q-determinant. Any numerical algorithm has a Q-determinant. The Qdeterminant of the algorithm has clear structure and is convenient for implementation. The Q-determinant consists of Q-terms. Their number is equal to the number of output data items. Each Q-term describes all possible ways to compute one of the output data items based on the input data.We also describe a software Q-system for studying the parallelism resource of numerical algorithms. This system enables to compute and compare the parallelism resources of numerical algorithms. The application of the Q-system is shown on the example of numerical algorithms with different structures of Q-determinants. Furthermore, we suggest a method for designing of parallel programs for numerical algorithms. This method is based on a representation of a numerical algorithm in the form of a Q-determinant. As a result, we can obtain the program using the parallelism resource of the algorithm completely. Such programs are called Q-effective.The results of this research can be applied to increase the implementation efficiency of numerical algorithms, methods, as well as algorithmic problems on parallel computing systems. CCS Concepts: • Theory of computation → Models of computation; Concurrency; Parallel computing models; Design and analysis of algorithms; Parallel algorithms; • Software and its engineering → Software creation and management; Software development techniques; Flowcharts; • Computing methodologies → Parallel computing methodologies; Parallel algorithms; Symbolic and algebraic manipulation; Symbolic and algebraic algorithms; Linear algebra algorithms;Additional Key Words and Phrases: Q-term of algorithm, Q-determinant of algorithm, representation of algorithm in form of Q-determinant, Q-effective implementation of algorithm, parallelism resource of algorithm, software Q-system, Q-effective program, Q-effective programming

show abstract

Section: Introductionmentioning

confidence: 99%

Parallelism Resource of Numerical Algorithms. Version 1

Aleeva¹,

Aleev²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Neighboring analysis applies to algorithms that are based on a fixed cell size and sliding analysis window, such as cellular automata [ 8 ], digital terrain analysis [ 9 ], and so on. Many research studies have been carried out on parallel implementation of raster-based spatial analysis via different parallel computing platforms.…”

Section: Introductionmentioning

confidence: 99%

A Parallel Computing Approach to Spatial Neighboring Analysis of Large Amounts of Terrain Data Using Spark

Zhang

Zheng

2021

Sensors

View full text Add to dashboard Cite

Spatial neighboring analysis is an indispensable part of geo-raster spatial analysis. In the big data era, high-resolution raster data offer us abundant and valuable information, and also bring enormous computational challenges to the existing focal statistics algorithms. Simply employing the in-memory computing framework Spark to serve such applications might incur performance issues due to its lack of native support for spatial data. In this article, we present a Spark-based parallel computing approach for the focal algorithms of neighboring analysis. This approach implements efficient manipulation of large amounts of terrain data through three steps: (1) partitioning a raster digital elevation model (DEM) file into multiple square tile files by adopting a tile-based multifile storing strategy suitable for the Hadoop Distributed File System (HDFS), (2) performing the quintessential slope algorithm on these tile files using a dynamic calculation window (DCW) computing strategy, and (3) writing back and merging the calculation results into a whole raster file. Experiments with the digital elevation data of Australia show that the proposed computing approach can effectively improve the parallel performance of focal statistics algorithms. The results also show that the approach has almost the same calculation accuracy as that of ArcGIS. The proposed approach also exhibits good scalability when the number of Spark executors in clusters is increased.

show abstract