Crop Classification Under Varying Cloud Cover With Neural Ordinary Differential Equations

Abstract

Optical satellite sensors cannot see the earth's surface through clouds. Despite the periodic revisit cycle, image sequences acquired by earth observation satellites are, therefore, irregularly sampled in time. State-of-the-art methods for crop classification (and other time-series analysis tasks) rely on techniques that implicitly assume regular temporal spacing between observations, such as recurrent neural networks (RNNs). We propose to use neural ordinary differential equations (NODEs) in combination with RNNs to classify crop types in irregularly spaced image sequences. The resulting ODE-RNN models consist of two steps: an update step, where a recurrent unit assimilates new input data into the model's hidden state, and a prediction step, in which NODE propagates the hidden state until the next observation arrives. The prediction step is based on a continuous representation of the latent dynamics, which has several advantages. At the conceptual level, it is a more natural way to describe the mechanisms that govern the phenological cycle. From a practical point of view, it makes it possible to sample the system state at arbitrary points in time such that one can integrate observations whenever they are available and extrapolate beyond the last observation. Our experiments show that ODE-RNN, indeed, improves classification accuracy over common baselines, such as LSTM, GRU, temporal convolutional network, and transformer. The gains are most prominent in the challenging scenario where only few observations are available (i.e., frequent cloud cover). Moreover, we show that the ability to extrapolate translates to better classification performance early in the season, which is important for forecasting.

Crop Yield Prediction using Machine Learning and Deep Learning Techniques

“…Therefore, the monitoring of the range of factors allows the farmers to develop strategic procedures for increasing the agricultural productivity of the nations. The usage of machine learning has refined the power of predicting crop yield and the presence of statistical models has helped the extrapolation of the available data from the past [16]. With the help of intelligent computational methods, the improvement of crop yield predictive analysis can be performed by the farmers and the researchers.…”

“…The use of gated mechanisms of GRU is necessary for selectively passing and updating the information present in the hidden layers in DL [25]. Such a measure takes into consideration the extent of the flow of information through the [16]. In the former case of the reset gate, the determination of the extent to which the previously hidden layer is to be omitted is inspected.…”

U-TILISE: A Sequence-to-Sequence Model for Cloud Removal in Optical Satellite Time Series

“…Instead, we encode the observation date day(t), expressed as the number of days since the 1 st of January of the respective calendar year. This strategy has proved beneficial for learned time series processing [2], [75], since it preserves information about seasonal patterns (e.g., lighting conditions or phenology of the vegetation) and accounts for irregular temporal sampling.…”

Deep Learning and Earth Observation to Support the Sustainable Development Goals: Current approaches, open challenges, and future opportunities

“…A recent alternative to RNN approaches for crop mapping involves neural ordinary differential equations, which can interpolate in the case of missing data [127]-due cloud coverage, for example. Finally, recent approaches have considered spatiotemporal bidirectional long short-term memory (bi-LSTM) architectures to fully exploit the information of long time series of high-resolution Sentinel-2 data to classify different crop types (rice, fallow, barley, oat, wheat, sunflower, and triticale) [121] (see Figure 4).…”