A Convolutional Neural Network Architecture for Sentinel-1 and AMSR2 Data Fusion

Abstract

With a growing number of different satellite sensors, data fusion offers great potential in many applications. In this work, a convolutional neural network (CNN) architecture is presented for fusing Sentinel-1 synthetic aperture radar (SAR) imagery and the Advanced Microwave Scanning Radiometer 2 (AMSR2) data. The CNN is applied to the prediction of Arctic sea ice for marine navigation and as input to sea ice forecast models. This generic model is specifically well suited for fusing data sources where the ground resolutions of the sensors differ with orders of magnitude, here 35 km × 62 km (for AMSR2, 6.9 GHz) compared with the 93 m × 87 m (for sentinel-1 IW mode). In this work, two optimization approaches are compared using the categorical cross-entropy error function in the specific application of CNN training on sea ice charts. In the first approach, concentrations are thresholded to be encoded in a standard binary fashion, and in the second approach, concentrations are used as the target probability directly. The second method leads to a significant improvement in R 2 measured on the prediction of ice concentrations evaluated over the test set. The performance improves both in terms of robustness to noise and alignment with mean concentrations from ice analysts in the validation data, and an R 2 value of 0.89 is achieved over the independent test set. It can be concluded that CNNs are suitable for multisensor fusion even with sensors that differ in resolutions by large factors, such as in the case of Sentinel-1 SAR and AMSR2.

Sea Ice Concentration Estimation: Using Passive Microwave and SAR Data With a U-Net and Curriculum Learning

“…Rough water conditions from wind and ocean currents can increase the backscatter. Previous studies have shown that deep learning models are a suitable choice for estimating ice concentration from SAR because they can learn backscatter patterns of ice and water [4], [5], [6]. The present study builds upon the previous work of using deep learning models for ice concentration estimation.…”

“…Recently, CNNs have shown significant potential for sea ice concentration estimation from SAR [4], [5], [6]. A CNN is a deep learning model that consists of both convolutional and fully connected layers.…”

“…Convolutional neural networks have been successful in estimating ice concentration when using image analysis chart estimates as training labels [4], [6]. Passive microwave data has also been used as training labels for a DenseNet model, a CNN with direct connections between all layers [11], to estimate ice concentration on the Gulf of St. Lawrence and Arctic Archipelago regions [5].…”

“…The limitation of the aforementioned CNNs for ice concentration estimation is that they make predictions for a single pixel each iteration. A parallel study utilized atrous convolutions with spatial pyramid pooling for ice concentration estimation, which allowed multi-dimensional predictions to be made in one pass and eliminated the need for downscaling [6]. In the present study, we follow a different approach to make multi-dimensional predictions that utilizes a U-net.…”

Baltic Sea Ice Concentration Estimation From C-Band Dual-Polarized SAR Imagery by Image Segmentation and Convolutional Neural Networks

“…From such a classification, also, SIC estimates can be derived. Recently, a method combining CNN applied to SAR and upsampled MWR brightness temperatures at the final layer of the neural network with a sigmoid activation has been proposed in [38].…”