Lightweight Temporal Self-attention for Classifying Satellite Images Time Series

Abstract

The increasing accessibility and precision of Earth observation satellite data offers considerable opportunities for industrial and state actors alike. This calls however for efficient methods able to process time-series on a global scale. Building on recent work employing multi-headed self-attention mechanisms to classify remote sensing time sequences, we propose a modification of the Temporal Attention Encoder of Garnot et al. [5]. In our network, the channels of the temporal inputs are distributed among several compact attention heads operating in parallel. Each head extracts highly-specialized temporal features which are in turn concatenated into a single representation. Our approach outperforms other state-of-the-art time series classification algorithms on an open-access satellite image dataset, while using significantly fewer parameters and with a reduced computational complexity.

Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series

“…Multiple recent studies [6,[17][18][19][20] have solidified the PSE+LTAE (Pixel Set Encoder and Lightweight Temporal Attention) as the state-of-the-art of crop type classification. Furthermore, this network is particularly parsimonious in terms of computation and memory usage, which proves well suited for training on multi-year data.…”

“…Its architecture is inspired by set-encoding deep architecture [13,37], and dispense us from preprocessing parcels into image patches, saving memory, and computation. The Temporal Attention Encoder (TAE) [6] and its parsimonious version Lightweight-TAE (LTAE) [18] are temporal sequence encoders based on the language processing literature [38] and adapted for processing SITS. Both networks can be used sequentially to map the sequence of observations x i at year i to a learned yearly spatiotemporal descriptor e i :…”

“…Single-Year: M single . We simply do not provide the labels of previous years, and directly map the current year's observations to a vector of class scores [18].…”

“…In contrast, the model M mixed is trained with all parcels across all years with no information regarding of the year of acquisition. All models share the same PSE+LTAE configuration [18]. We visualize the training protocols in Figure 5, and report the results in Table 2.…”

“…However, this scenario is not compatible with the operational setting of crop monitoring, in which payment agencies may want to detect erroneous declarations before all farmers' declarations have been received. Instead, we use the same setting than the vast majority of work in parcel classification and whose task is to classify parcels after the year is over [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20].…”