A Topology Based Spatio-Temporal Map Algebra for Big Data Analysis

Abstract

Continental and global datasets based on earth observations or computational models challenge the existing map algebra approaches. The available datasets differ in their spatio-temporal extents and their spatio-temporal granularity, which makes it difficult to process them as time series data in map algebra expressions. To address this issue we introduce a new map algebra approach that is topology based. This topology based map algebra uses spatio-temporal topological operators (STTOP and STTCOP) to specify spatio-temporal operations between topological related map layers of different time-series data. We have implemented several topology based map algebra tools in the open source geoinformation system GRASS GIS and its open source cloud processing engine actinia. We demonstrate the application of our topology based map algebra by solving real world big data problems using a single algebraic expression. This included the massively parallel computation of the NDVI from a series of 100 Sentinel2A scenes organized as earth observation data cubes. The processing was performed and benchmarked on a many core computer setup and in a distributed container environment. The design of our topology based map algebra allows us to deploy it as a standardized service in the EU Horizon 2020 project openEO.

On-Demand Processing of Data Cubes from Satellite Image Collections with the gdalcubes Library

“…For example, if a data cube pixel has a temporal duration of one month, values from multiple images need to be combined, e.g., by averaging the five-daily values covered by a particular month. Similar to [21], who formalize a topological map algebra for analyzing irregular spatiotemporal datasets including satellite image collections, this allows to adapt the temporal granularity to the specific needs, and to make this explicit.…”

“…The only other software systems we know of that can create regular data cubes from image collections are GRASS GIS [21], Open Data Cube [11], and the (non-open source) Google Earth Engine [3]. The open source library gdalcubes introduced here is a nice addition to these as it is relatively easy to integrate in scripting languages such as R, Julia, or Python, and can work in conjunction with software that can process data cubes such as GRASS GIS [32], R packages raster [16] and stars [17], and Python packages numpy [33] and xarray [13].…”

Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes

“…To benefit from the large volume of EO data made available with Data Cubes, recent Open Source developments were allowed to implement solutions in open source geoinformation and statistical software. Gebbert et al [38] have developed spatio-temporal topological operators in the GRASS GIS software to enable the effective use of heterogenous (e.g., extent, granularity) spatio-temporal EO data. Similarly, Appel et al [39] introduced an open source C++ library and R package for the construction and processing of on-demand data cubes from satellite image collections, and showed how it supports interactive method development workflows where data users can initially try methods on small subsamples before running analyses on high resolution and/or large areas.…”