Multimodal Deep Learning for Heterogeneous GNSS-R Data Fusion and Ocean Wind Speed Retrieval

Abstract

The comprehensiveness of the raw input data and the effectiveness of feature engineering are two key factors affecting the performance of machine learning (ML). To improve the data comprehensiveness for Global Navigation Satellite System Reflectometry (GNSS-R) ocean wind speed retrieval, this paper introduces a new input data structure, which is composed of Delay-Doppler Maps (DDM) and all satellite receiver status (SRS) parameters. Then, to overcome the difficulty of handcrafted feature engineering and effectively fusion the information of DDM and SRS, we presented a heterogeneous multimodal deep learning (HMDL) method to retrieve the wind speed according to the heterogeneity of the input data. The proposed model is verified by the performance evaluation of realistic data sets obtained from TDS-1. The new input data structure improves the prediction accuracy at 13.5% to 30.7% on mean absolute error (MAE) at 10.6% to 29.5% on the root mean square error (RMSE). The HMDL improves the prediction accuracy at 7.7% on MAE and 7.1% on RMSE. The whole proposed solution improves the prediction accuracy at 36.3% on MAE and 36.8% on RMSE, comparing with the traditional neural network (NN) based solution. The results clearly show that both the introduction of the new input data structure and HMDL effectively improve the accuracy and robustness of GNSS-R wind speed retrieval.

Retrieval of the Characteristic Size of Raindrops for Wind Sensing Based on Dual-Polarization Radar

“…After basic quality control of the data (for example, Z DR is limited in [0. 5,3] which is a value range of Z DR in previous studies [33], [34], [48] and in our simulation), D R was calculated with Z DR and , can be obtained from the retrieval models based on Z DR , D R and the fusion models of Z DR + D R , respectively. For a certain elevation angle at each time frame, the RMSE of retrieved D c is shown in Fig.…”

“…In (5), the part that determines the density of the spectrum is defined as equivalent RCS σ e , as shown in (6):…”

FA-RDN: A Hybrid Neural Network on GNSS-R Sea Surface Wind Speed Retrieval

“…This quantification process is usually implemented by defining a loss function. The loss function selected in proposed model is the same as the one used in previous studies, which is the mean square error (MSE) function [25]. The training process of the neural network is briefly described as updating the weights and biases of each layer of the model to minimize the difference between the output result and the real value.…”

“…This quantification process is usually implemented by defining a loss function. The loss function selected in proposed model is the same as the one used in previous studies, which is the mean square error (MSE) function [25].…”

“…Y. Liu et al built a multi-hidden-layer neural network with deeper network layers to invert wind speed by CYGNSS data and obtained more accurate results compared with SVM and RF [24]. In addition, X. Chu et al had also explored wind speed retrieval with TDS-1 data using deep learning [25].…”