Parallel Spatial-Data Conversion Engine: Enabling Fast Sharing of Massive Geospatial Data

Abstract

Large-scale geospatial data have accumulated worldwide in the past decades. However, various data formats often result in a geospatial data sharing problem in the geographical information system community. Despite the various methodologies proposed in the past, geospatial data conversion has always served as a fundamental and efficient way of sharing geospatial data. However, these methodologies are beginning to fail as data increase. This study proposes a parallel spatial data conversion engine (PSCE) with a symmetric mechanism to achieve the efficient sharing of massive geodata by utilizing high-performance computing technology. This engine is designed in an extendable and flexible framework and can customize methods of reading and writing particular spatial data formats. A dynamic task scheduling strategy based on the feature computing index is introduced in the framework to improve load balancing and performance. An experiment is performed to validate the engine framework and performance. In this experiment, geospatial data are stored in the vector spatial data defined in the Chinese Geospatial Data Transfer Format Standard in a parallel file system (Lustre Cluster). Results show that the PSCE has a reliable architecture that can quickly cope with massive spatial datasets. . Moreover, semantic heterogeneity often occurs at the information level, requiring systems to understand the content of an item of information and its meaning, that is, semantics [9,10].Despite such obstacles, the reusability of existing data is still essential simply because of the high cost of acquiring new geographical data from scratch. Many researchers have presented various approaches that enable spatial data sharing and example systems [6,[8][9][10][11][12][13][14][15][16][17][18][19][20]. These approaches may be classified into three general categories, namely, spatial data conversion, spatial federated database, and mediator-based data sharing.Spatial data conversion techniques are the basic methods for spatial data sharing. They can be traced to the beginning of the GIS industry when a huge amount of spatial data were collected in isolated systems or departments, and the need for data sharing increased among geospatial data producers. People have attempted to convert spatial data from data sources directly into their own systems to overcome data sharing problems [8,9,14]. Currently, spatial data conversion techniques have developed extremely well and gained the power to transfer between two formats in over a hundred cases. However, such techniques often cause data redundancy and result in improper stress when coping with large datasets.Federated database techniques soon took the focus because they had several advantages over data conversion methodologies. They have allowed each database owner to define the subsets of local data and then integrate them into one virtual database by establishing a global schema in a common data model, thereby allowing users to access the federated database like a centralized database, without having...

Prediction model of mobike demand based on T-GCN

“…The value of seq_len is important. In this experiment, we choose the value of seq_len from [1,2,4,8,16,24,32,36,48,64,72]. 80% of the data is used as the training set and 20% as the testing set.…”

Critical success factors for geospatial data sharing in disaster management

“…Critical success factors in disaster geospatial data sharing In a recent study by [23], geospatial data sharing issues are among the main areas for planning and making the best decisions in a public emergency to get a complete picture of the situation, expedite scientific research and studies, and save time making decisions. Although most of the data involving geospatial is stored by the authorities, the sharing practices carried out have been found to be ineffective and hindered or slowed down by the lack of clear elements of support for its implementation.…”

“…Although most of the data involving geospatial is stored by the authorities, the sharing practices carried out have been found to be ineffective and hindered or slowed down by the lack of clear elements of support for its implementation. Geospatial data exchange on a large scale amongst companies in the GIS community has long been a challenge [24]. Based on past studies and interviews with the data provider there are thirteen (13) factors that have caused the sharing of geospatial data to be implemented comprehensively.…”

Parallel Hydrological Model Parameter Uncertainty Analysis Based on Message-Passing Interface

“…The hardware environment mainly includes the multicore central processing unit (CPU), graphics processing unit (GPU) and computer cluster, while the software environment mainly includes the Message-Passing Interface (MPI), OpenMP, Compute Unified Device Architecture (CUDA, NVIDIA Corporation, Santa Clara, CA, USA) and MapReduce. Since parallel computing was proposed, it has been applied in many fields, such as transportation [30], geology [31], information science [32], geospatial algorithms [33,34], etc.…”