Unlocking the Australian Landsat Archive – From dark data to High Performance Data infrastructures

Building an Earth Observations Data Cube: lessons learned from the Swiss Data Cube (SDC) on generating Analysis Ready Data (ARD)

“…space, time, data type) stack of spatially aligned pixels ready for analysis. The concept has been proven and validated by Geoscience Australia together with CSIRO and the National Computing Infrastructure of Australia (NCI) who implemented the Australian Geoscience Data Cube, a national/continental scale DC of thousands of terabytes of EO data (Landsat, MODIS, Sentinel-2) making it quicker and easier to provide information on environmental issues that can affect all Australians Lewis et al, 2016;Purss et al, 2015). It has allowed mapping the extent of surface water across the entire Australian continent using 27 years of Landsat imagery (Mueller et al, 2016), gaining knowledge on flood dynamics over Australia (Tulbure, Broich, Stehman, & Kommareddy, 2016), or extracting the intertidal extent and topography of the Australian coastline (Sagar, Roberts, Bala, & Lymburner, 2017).…”

“…They remain still underutilized and stored in electronic silos of data (Gore, 1998;Lewis et al, 2016). This is due to several reasons: (1) increasing volumes of data generated by EO satellites; (2) lack of expertise, infrastructure, or internet bandwidth to efficiently and effectively access, process, and utilize EO data; (3) the particular type of highly structured data that EO data represent introducing challenges when trying to integrate or analyze them; (4) and the substantial effort and cost required to store and process data limits the efficient use of these data (CEOS, 2017;Lewis et al, 2016;Purss et al, 2015). Therefore, EO data can be considered as Big Data, data that are too large, fast-lived, heterogeneous, or complex to get understood and exploited (Baumann, Rossi, et al, 2016).…”

“…scene-based file download) and lower the barriers caused by data size and related complexities in data preparation, handling, storage and analysis (CEOS, 2017). This paradigm shift is currently represented by EO Data Cubes (Baumann, Mazzetti, et al, 2016;Purss et al, 2015), an approach that is receiving increasing attention as a new solution to store, organize, manage, and analyze EO data in a way that was not possible before. Data Cubes (DC) are aiming to realize the full potential of EO data repositories by addressing Volume, Velocity, and Variety challenges, providing access to large spatio-temporal data in an analysis ready form (Baumann, 2017;Lewis et al, 2017).…”

Explaining and Measuring Social-Ecological Pathways: The Case of Global Changes and Water Security

“…EO data can provide an efficient and effective mechanism to support water security monitoring. However, the full potential information of EO data is difficult to realize mainly because of their complexity, increasing volume, and the lack of efficient processing capabilities [76,77]. Consequently, new approaches are required to rapidly analyze data in a transparent and repeatable manner and to facilitate the transformation of many EO data into actionable information and decision-ready products [75].…”

Advancements in Open Geospatial Standards for Photogrammetry and Remote Sensing From Ogc

“…For more information see (WCPS, 2016) Use of WCS with array databases and Big Data applications is a rich and innovative area of application. Two excellent examples are Geoscience Australia Data Cube (Purss, 2015) and EarthServer (Baumann, 2016) projects.…”