The R-based climate4R open framework for reproducible climate data access and post-processing

Iturbide, Maialen; Bedia, Joaquín; Herrera, Sixto; Baño‐Medina, Jorge; Fernández, Jesús Fernández; Frías, M. D. Rodríguez; Manzanas, Rodrigo; San-Martín, Daniel; Cimadevilla, Ezequiel; Cofiño, Antonio S.; Gutiérrez, José Manuel

doi:10.1016/j.envsoft.2018.09.009

Cited by 111 publications

(93 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the purpose of research transparency, we provide the full code needed to reproduce the experiments presented in this paper, which can be found in the Santander Meteorology Group GitHub (https://github.com/SantanderMetGroup/DeepDownscaling) and in Zenodo (Baño Medina et al, 2019). The code builds on the open-source climate4R (Iturbide et al, 2019) and keras (Chollet et al, 2015) R frameworks, for the benchmark and the CNN models, respectively. The former is an open R framework for climate data access, processing (e.g.…”

Section: Discussionmentioning

confidence: 99%

Configuration and Intercomparison of Deep Learning Neural Models for Statistical Downscaling

Baño‐Medina¹,

Gutiérrez²

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. Deep learning techniques (in particular convolutional neural networks, CNNs) have recently emerged as a promising approach for statistical downscaling due to their ability to learn spatial features from huge spatio-temporal datasets. However, existing studies are based on complex models, applied to particular case studies and using simple validation frameworks, which makes difficult a proper assessment of the (possible) added value offered by these techniques. As a result, these models are usually seen as black-boxes generating distrust among the climate community, particularly in climate change problems. In this paper we undertake a comprehensive assessment of deep learning techniques for continental-scale statistical downscaling, building on the VALUE validation framework. In particular, different CNN models of increasing complexity are applied for downscaling temperature and precipitation over Europe, comparing them with a few standard benchmark methods from VALUE (linear and generalized linear models) which have been traditionally used for this purpose. Besides analyzing the adequacy of different components and topologies, we also focus on their extrapolation capability, a critical point for their possible application in climate change studies. To do this, we use a warm test period as surrogate of possible future climate conditions. Our results show that, whilst the added value of CNNs is mostly limited to the reproduction of extremes for temperature, these techniques do outperform the classic ones for the case of precipitation for most aspects considered. This overall good performance, together with the fact that they can be suitably applied to large regions (e.g. continents) without worrying about the spatial features being considered as predictors, can foster the use of statistical approaches in international initiatives such as CORDEX.

show abstract

Section: Discussionmentioning

confidence: 99%

Configuration and Intercomparison of Deep Learning Neural Models for Statistical Downscaling

Baño‐Medina¹,

Gutiérrez²

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In bias adjustment, the R package Climate4R (Iturbide et al, 2018) was utilised as it enables rapid testing of different bias-correction methods. Climate4R provides bias-correction methods for both temperature (and similar variables) and precipitation.…”

Section: Methodsmentioning

confidence: 99%

“…We concentrated on reforecasts of soil moisture available from ECMWF SEAS5 seasonal forecast system (Johnson et al, 2019) for years 1981-2016. Data was accessed through the Meteorological Archival and Retrieval System (MARS).…”

Section: Datamentioning

confidence: 99%

Bias-adjusted seasonal forecasts of soil moisture for forestry applications in Finland

2020

View full text Add to dashboard Cite

Seasonal forecasts for forestry have been developed in the Finnish Meteorological Institute in cooperation with Finnish end-users. Such forecasts could help forest companies in preparing for adverse conditions of timber harvesting operations. Bearing capacity for harvesting operations is dependent on soil moisture, and skillful forecasts have potentially large economic value. Using the ECMWF seasonal forecasts, we evaluated the monthly mean soil moisture forecasts for four different start months, with lead times from 0 to 2 months. Forecasts were bias adjusted, and showed skill in the first month for all four months. After the first lead month, winter months fared a bit better than summer months.

show abstract

“…There are also domain-specific frameworks and toolkits. For instance, [65] introduces an R-based framework called climate4R that aims to standardize the tools used for accessing, harmonizing and post-processing climate data. It provides access to both local and remote data and features built-in support for a wide range of remote data sources.…”

Section: Recomputation and Reproducibilitymentioning

confidence: 99%

State of the Art and Future Trends in Data Reduction for High-Performance Computing

2020

JSFI

View full text Add to dashboard Cite

Research into data reduction techniques has gained popularity in recent years as storage capacity and performance become a growing concern. This survey paper provides an overview of leveraging points found in high-performance computing (HPC) systems and suitable mechanisms to reduce data volumes. We present the underlying theories and their application throughout the HPC stack and also discuss related hardware acceleration and reduction approaches. After introducing relevant use-cases, an overview of modern lossless and lossy compression algorithms and their respective usage at the application and file system layer is given. In anticipation of their increasing relevance for adaptive and in situ approaches, dimensionality reduction techniques are summarized with a focus on non-linear feature extraction. Adaptive approaches and in situ compression algorithms and frameworks follow. The key stages and new opportunities to deduplication are covered next. An unconventional but promising method is recomputation, which is proposed at last. We conclude the survey with an outlook on future developments.

show abstract

The R-based climate4R open framework for reproducible climate data access and post-processing

Cited by 111 publications

References 35 publications

Configuration and Intercomparison of Deep Learning Neural Models for Statistical Downscaling

Configuration and Intercomparison of Deep Learning Neural Models for Statistical Downscaling

Bias-adjusted seasonal forecasts of soil moisture for forestry applications in Finland

State of the Art and Future Trends in Data Reduction for High-Performance Computing

Contact Info

Product

Resources

About