Ocean Current Prediction Using the Weighted Pure Attention Mechanism

Abstract

Ocean current (OC) prediction plays an important role for carrying out ocean-related activities. There are plenty of studies for OC prediction with deep learning to pursue better prediction performance, and the attention mechanism was widely used for these studies. However, the attention mechanism was usually combined with deep learning models rather than purely used to predict OC, or, if it was purely used, did not further optimize the attention weight. Therefore, a deep learning model based on weighted pure attention mechanism is proposed in this paper. This model uses the pure attention mechanism, introduces a weight parameter for the generated attention weight, and moves more attentions from other elements to the key elements based on weight parameter setting. To our knowledge, it is the first attempt to use the weighted pure attention mechanism to improve the OC prediction performance, and it is an innovation for OC prediction. The experiment results indicate that the proposed model can fully take advantage of the strengths from the pure attention mechanism; it can further optimize the pure attention mechanism and significantly improve the prediction performance, and is reliable for OC prediction with high performance for a wide time range and large spatial scope.

Short-Term Exhaust Gas Temperature Trend Prediction of a Marine Diesel Engine Based on an Improved Slime Mold Algorithm-Optimized Bidirectional Long Short-Term Memory—Temporal Pattern Attention Ensemble Model

“…It allows the model to focus on processing information relevant to specific tasks, similar to how the human brain processes information. This mechanism demonstrates versatile application potential in various areas such as language processing, image analysis, and prediction tasks [27][28][29]. It enables models to deeply utilize crucial information from historical data, thereby enhancing the ability to recognize and utilize key patterns.…”

ASTMEN: an adaptive spatiotemporal and multi-element fusion network for ocean surface currents forecasting

“…Thongniran et al used convolutional neural networks (CNN) and GRU to construct a two-stage CNN-GRU model for spatial and temporal forecasting of ocean currents (Thongniran et al, 2019). Liu et al used a singlestage ConvLSTM to fuse spatial information for ocean currents at any given moment, and the forecast accuracy is better than the twostage approach (Liu et al, 2022). However, the fixed convolutional kernel in CNNs makes it difficult for these methods to address situations where different patterns of currents occur simultaneously and the patterns of currents change over time (Özturk et al, 2018;Gu et al, 2023).…”

“…In order to evaluate the performance of ASTMEN in forecasting ocean surface currents, five algorithms are compared, of which LSTM, CNN-GRU and ConvLSTM have been shown to be effective for spatial and temporal forecasting of ocean surface currents (Thongniran et al, 2019;Immas et al, 2021;Liu et al, 2022). This study also adds the climate state calculated from the CORA 2.0 reanalysis dataset during 2015-2018 as a comparison, to verify the effectiveness of ASTMEN.…”