Comprehensive Framework of Early and Late Fusion for Image–Sentence Retrieval

Wang, Yifan; Xu, Xing; Yu, Wu; Xu, Ruicong; Cao, Zhiwei; Shen, Heng Tao

doi:10.1109/mmul.2022.3144972

Cited by 5 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Late fusion can capture the temporal correlation and interaction effects between data sources well and has better performance in time series tasks [30]. However, the above two methods have the shortcomings of a complex model structure, a high demand for computational resources, and a high cost of training time when the number of samples is small [31]. Therefore, for the discrete small-sample ship engine failure sample set, this paper chooses early fusion [32] as the method of fusion of the multi-graph structure, sets the feature weight W i and neighbor threshold T of each graph to construct the fusion graph structure, and its formula is as follows:…”

Section: Feature Graph Fusionmentioning

confidence: 99%

Research Method for Ship Engine Fault Diagnosis Based on Multi-Head Graph Attention Feature Fusion

Ai,

Cao,

Wang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

At present, there are problems such as low fault data, insufficient labeling information, and poor fault diagnosis in the field of ship engine diagnosis. To address the above problems, this paper proposes a fault diagnosis method based on probabilistic similarity and rank-order similarity of multi-head graph attention neural networks (MPGANN) models. Firstly, the ship engine dataset is used to explore the similarity between the data using the probabilistic similarity of T_SNE and the rank order similarity of Spearman’s correlation coefficient to define the neighbor relationship between the samples, and then the appropriate weights are selected for the early fusion of the two graph structures to fuse the feature information of the two scales. Finally, the graph attention neural networks (GANN) incorporating the multi-head attention mechanism are utilized to complete the fault diagnosis. In this paper, comparative experiments such as graph construction and algorithm performance are carried out based on the simulated ship engine dataset, and the experimental results show that the MPGANN outperforms the comparative methods in terms of accuracy, F1 score, and total elapsed time, with an accuracy rate of 97.58%. The experimental results show that the model proposed in this paper can still fulfill the ship engine fault diagnosis task well under unfavorable conditions such as small samples and insufficient label information, which is of practical significance in the field of intelligent ship cabins and fault diagnosis.

show abstract

Section: Feature Graph Fusionmentioning

confidence: 99%

Research Method for Ship Engine Fault Diagnosis Based on Multi-Head Graph Attention Feature Fusion

Ai,

Cao,

Wang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

A review of aquaculture: From single modality analysis to multimodality fusion

Li,

Du,

et al. 2024

Computers and Electronics in Agriculture

View full text Add to dashboard Cite

Ship Ballast Water System Fault Diagnosis Method Based on Multi-Feature Fusion Graph Convolution

Ai,

Cao,

Wang

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

To tackle the issues of limited fault data, inadequate information availability, and subpar fault diagnosis within the realm of ship ballast water system condition monitoring, this paper presents a novel fault diagnosis methodology known as the Probabilistic Similarity and Linear Similarity-based Graph Convolutional Neural Network (PCGCN) model. PCGCN initially converts the ship’s ballast water system dataset into two distinct graph structures: a probabilistic topology graph and a correlation topology graph. It delves into data similarity by employing T-SNE for probabilistic similarity and Pearson’s correlation coefficient for linear similarity to establish the inter-sample neighbor relationships. Subsequently, an early fusion of these two graph structures is conducted to extract more profound multi-scale feature information. Following this step, the graph convolutional neural network (GCN) is introduced to amalgamate the feature information from neighboring nodes in addition to its inherent features. This is aimed at enhancing the available information for the classification task and addressing the issues of limited fault data and inadequate label information. In conclusion, we employ a simulated ship fault dataset for testing experiments, and the PCGCN model demonstrates superior classification accuracy, reaching 97.49%, outperforming traditional diagnostic methods. These experimental outcomes underscore the applicability of the model introduced in this study to the realm of ship ballast water system fault diagnosis, even under challenging conditions characterized by limited sample sizes and insufficient labeling information.

show abstract

Comprehensive Framework of Early and Late Fusion for Image–Sentence Retrieval

Cited by 5 publications

References 16 publications

Research Method for Ship Engine Fault Diagnosis Based on Multi-Head Graph Attention Feature Fusion

Research Method for Ship Engine Fault Diagnosis Based on Multi-Head Graph Attention Feature Fusion

A review of aquaculture: From single modality analysis to multimodality fusion

Ship Ballast Water System Fault Diagnosis Method Based on Multi-Feature Fusion Graph Convolution

Contact Info

Product

Resources

About