Modeling vegetation response to climate in Africa at fine resolution: EarthNet2023, a deep learning dataset and challenge.

Abstract

<p>Droughts are a major disaster in Africa, threatening livelihoods through their influence on crop yields but also by impacting and weakening ecosystems. Modeling the vegetation state can help anticipate and reduce the impact of droughts by predicting the vegetation response over time. Forecasting the state of vegetation is challenging: it depends on complex interactions between the plants and different environmental drivers, which can result in both instantaneous and time-lagged responses, as well as spatial effects. Furthermore, modeling these interactions at the fine resolution of landscape scale can only rely on remote sensing observations, as in-situ measurements are not global and weather models have a coarse grid. With the increasing availability of remote sensing data, deep learning methods are a promising avenue for these spatiotemporal tasks. Here, we introduce both a dataset and a baseline deep neural network, modeling the vegetation response to climate at landscape scale in Africa.</p> <p>EarthNet2021 [1] introduced leveraging self-supervised learning for satellite imagery forecasting based on coarse-scale weather in Europe. Here, we introduce EarthNet2023 with a more narrow focus on drought impacts in Africa. It contains over 45,000 Spatio-temporal minicubes (each 1.28x1.28km) at representative locations over the whole African continent. Alongside Sentinel-2 reflectance, ERA5 weather, and topography, it also contains Sentinel-1 backscatter, soil properties, and a long-term Normalized Difference Vegetation Index (NDVI) climatology based on Landsat. The latter allows evaluating models on vegetation anomalies, thereby including modeling of drought impacts. EarthNet2023 is intended as an open benchmark challenge, allowing multiple research groups to develop their approaches to drought impact modeling in Africa. </p> <p>As a baseline for EarthNet2023, we train a  Convolutional Long Short-Term Memory (ConvLSTM) deep learning model. Previous work has shown it is suitable for spatiotemporal satellite imagery forecasting [2, 3, 4]. The ConvLSTM baseline captures the seasonal evolution of NDVI over a wide range of vegetation types. General spatial patterns are well-captured as well as a first indication of skill during weather extremes is seen, although the accuracy of the predictions is inconsistent, and the confidence in the model is therefore too low. This suggests, with further development, deep learning approaches are promising for modeling vegetation evolution in Africa, potentially even up to the degree to support anticipatory action with drought impact modeling.</p> <p> </p> <p>[1] Requena-Mesa, C., Benson, V., Reichstein, M., Runge, J., & Denzler, J. (2021). EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task. In <em>CVPR 2021</em> (pp. 1132-1142).</p> <p>[2] Diaconu, C. A., Saha, S., Günnemann, S., & Zhu, X. X. (2022). Understanding the Role of Weather Data for Earth Surface Forecasting Using a ConvLSTM-Based Model. In <em>CVPR 2022 </em>(pp. 1362-1371).</p> <p>[3] Kladny, K. R. W., Milanta, M., Mraz, O., Hufkens, K., & Stocker, B. D. (2022). Deep learning for satellite image forecasting of vegetation greenness. <em>bioRxiv</em>.</p> <p>[4] Robin, C., Requena-Mesa, C., Benson, V., Alonso, L., Poehls, J., Carvalhais, N., & Reichstein, M. (2022). Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs. In <em>Tackling Climate Change with Machine Learning: workshop at NeurIPS 2022.</em> </p>

Forecasting localized weather impacts on vegetation as seen from space with meteo-guided video prediction

“…It consists of two networks, each containing two ConvLSTM cells, without parameter sharing: One for the context period which works with past satellite imagery and past weather, and one for the target period, only using future weather as input. This is in contrast to the ConvLSTM flavors previously studied on EarthNet2021 [10,23], but has been shown to work better on a similar problem in Africa [45].…”

“…With EarthNet2021 [43], the first dataset for continental-scale satellite imagery forecasting was introduced. Subsequent works leveraged the Con-vLSTM model [50] for satellite imagery prediction [10,23] and for vegetation prediction in Africa [45]. Another line of work focuses on imputing cloudy time steps [31,63], yet often with a focus on historical gapfilling instead of nearrealtime information.…”

“…The persistence baseline, as in EarthNet2021 [43], constantly predicts the last valid NDVI observation from the context period. The previous year baseline [45] predicts the NDVI that was observed one year ago, obtained by interpolating last years observations linearly. The climatology baseline is produced by interpolating the NDVI timeseries leaving out the desired target year, then taking the mean over years, and then smoothing with a one month box-filter (fig.…”

“…Forecasting optical satellite imagery as a way to tackle both issues has recently been investigated with video prediction methods [10,23,43,45] on the EarthNet2021 dataset [43]. These models are able to forecast satellite imagery of high perceptual quality.…”

Learning Extreme Vegetation Response to Climate Forcing: A Comparison of Recurrent Neural Network Architectures

“…However, these models exhibited limited predictive capability for extreme events. Spatial information has proven helpful in similar investigations (Requena-Mesa et al, 2021;Diaconu et al, 2022;Kladny et al, 2022;Robin et al, 2022;Benson et al, 2023) and it could be beneficial to explore the extent to which it contributes to extreme conditions. Furthermore, the models used could be tailored more to the task.…”