Semantics lead all: Towards unified image registration and fusion from a semantic perspective

Xie, Housheng; Zhang, Yukuan; Qiu, Junhui; Zhai, Xiangshuai; Liu, Xuedong; Yang, Yang; Zhao, Shan; Luo, Yongfang; Zhong, Jie

doi:10.1016/j.inffus.2023.101835

Cited by 34 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convolutional neural networks have become the first choice of image fusion methods as networks have evolved. The fusion method of infrared and visible images also uses convolutional neural networks more widely [21][22][23][24][25][26]. The single-branch fusion method and two-branch fusion method are two kinds of convolutional-neural-network-based fusion methods.…”

Section: Convolutional-neural-network-based Fusion Methodsmentioning

confidence: 99%

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

et al. 2023

Self Cite

View full text Add to dashboard Cite

This study proposes a fusion method of infrared and visible images based on feature interaction. Existing fusion methods can be classified into two categories based on a single-branch network and a two-branch network. Generative adversarial networks are widely used in single-branch-based fusion methods, which ignore the difference in feature extraction caused by different input images. Most two-branch-based fusion methods use convolutional neural networks, which do not take into account the inverse promotion of fusion results and lack the interaction between different input features. To remedy the shortcomings of these fusion methods and better utilize the feature from source images, this study proposes a two-branch feature interactions method based on a generative adversarial network for visible and infrared image fusion. In the generator part, a two-branch feature interaction approach was designed to extract features from different inputs and realize feature interaction through the network connection of different branches. In the discriminator part, a double-classification discriminator was used for visible images and infrared images. Extensive comparison experiments with state-of-the-art methods have demonstrated the advantages of this proposed generative adversarial network based on two-branch feature interaction, which can enhance the texture details of objects in fusion results and reduce the interference of noise information from source inputs. In addition, the above advantages were also confirmed in generalization experiments of object detection.

show abstract

Section: Convolutional-neural-network-based Fusion Methodsmentioning

confidence: 99%

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Profile registration directly determines the measurement precision, so it has become the research emphasis of rail profile measurement. In recent decades, there are many studies on the registration problems [12,13], most of them take the profile to be measured and the standard model as two point-sets, and implement registration by solving the rigid transformation of them. ICP can be directly applied to pointsets without depending on the features of original data.…”

Section: Related Workmentioning

confidence: 99%

A Fast Global Optimal Strategy for Iteration Closest Point Using 2D‐BnB and Its Application to Rail Profile Registration

Jin,

2023

Journal of Sensors

View full text Add to dashboard Cite

Profile registration is critical to rail wear measurement with line structured light, and the most popular registration method is iteration closest point (ICP). Unfortunately, ICP is often invalid in actual applications because it is easy to trap into local minima. To solve this problem, we propose a hybrid 2D-point-set registration method which combined ICP to branch and bound. In this way, we can ensure that the ICP algorithm converges to the global optimum. This strategy can achieve high-registration precision, but it suffers from large computation costs. To address this issue, we propose an acceleration scheme by sparsely sampling the point-set before registration to relieve computation burden. Extensive experiments are conducted to verify the precision, stability, and efficiency of our method. The results show that our method has superior precision and stability compared to the other typical profile registration methods. The ability to achieve fast registration speed which is suitable for dynamic measurement is another contribution of our work.

show abstract

“…10,29 Deep learning-based methods, such as RFNet, AT-GAN, and SemLA, utilize deep neural networks to promote multimodal image registration accuracy. 15,36,37 However, deep learning-based methods depend greatly on the quality of training data, which limits their performance in the registration of coral reefs with less texture.…”

Section: Introductionmentioning

confidence: 99%

Automatic registration method for medium-resolution remote sensing images of coral reefs with morphological information pairing and constrained iterative fining

Chen,

Pian,

Chen

et al. 2023

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

Semantics lead all: Towards unified image registration and fusion from a semantic perspective

Cited by 34 publications

References 30 publications

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

Two-Branch Feature Interaction Fusion Method Based on Generative Adversarial Network

A Fast Global Optimal Strategy for Iteration Closest Point Using 2D‐BnB and Its Application to Rail Profile Registration

Automatic registration method for medium-resolution remote sensing images of coral reefs with morphological information pairing and constrained iterative fining

Contact Info

Product

Resources

About