Deep Seasonal Network for Remote Sensing Imagery Classification of Multi-Temporal Sentinel-2 Data

Abstract

As a medium-resolution multi-temporal data source, Sentinel-2 data has the potential to match the performance of using very-high-resolution (VHR) images in deep learning applications. To fully leverage the multi-temporal nature of Sentinel-2 data, we introduce the Deep Seasonal Network (DeepSN). This composite architecture combines a pre-trained deep convolutional neural network (DCNN) for visual feature extraction with a long short-term memory (LSTM) model to capture temporal information and make classification predictions. We evaluate the effectiveness of DeepSN on a Maasai Boma classification task in the Tanzania region. The DeepSN takes a sequence of four seasonal data, each spanning three months, for Boma prediction. Through cross-season validation experiments, we compare various advanced DCNNs and select EfficientNet as the backbone for DeepSN, as it performs the best. DeepSN with an EfficientNet backbone achieves a significant 19% improvement in the F1 score compared to plain EfficientNet for the Boma classification task. This work introduces a versatile composite architecture capable of handling multi-temporal data efficiently, providing flexibility in choosing the most suitable feature extraction backbone. The performance of DeepSN demonstrates the viability of utilizing medium-resolution multi-temporal data instead of high-resolution images for diverse tasks.