AEFusion: A multi-scale fusion network combining Axial attention and Entropy feature Aggregation for infrared and visible images

Li, Bicao; Lu, Ji-Qing; Liu, Zhoufeng; Shao, Zhuhong; Li, Chunlei; Du, Yifan; Huang, Jie

doi:10.1016/j.asoc.2022.109857

Cited by 15 publications

(15 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To comprehensively evaluate the proposed method, we perform qualitative and quantitative experiments on the MSRS dataset [ 7 ] with 361 image pairs, the LLVIP dataset [ 48 ] with randomly selected 389 image pairs, and the TNO dataset [ 49 ] with randomly selected 16 image pairs. We compare our method with eight state-of-the-art (SOTA) approaches, including DenseFuse [ 24 ], FusionGAN [ 26 ], SwinFuse [ 29 ], U2Fusion [ 50 ], AUIF [ 51 ], CUFD [ 23 ], MUFusion [ 52 ], and AEFusion [ 53 ]. The implementations of these approaches are publicly available and the initial parameters of the compared methods remain the same.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion

Li,

Xiao,

Cheng

et al. 2023

Sensors

View full text Add to dashboard Cite

The infrared and visible image fusion task aims to generate a single image that preserves complementary features and reduces redundant information from different modalities. Although convolutional neural networks (CNNs) can effectively extract local features and obtain better fusion performance, the size of the receptive field limits its feature extraction ability. Thus, the Transformer architecture has gradually become mainstream to extract global features. However, current Transformer-based fusion methods ignore the enhancement of details, which is important to image fusion tasks and other downstream vision tasks. To this end, a new super feature attention mechanism and the wavelet-guided pooling operation are applied to the fusion network to form a novel fusion network, termed SFPFusion. Specifically, super feature attention is able to establish long-range dependencies of images and to fully extract global features. The extracted global features are processed by wavelet-guided pooling to fully extract multi-scale base information and to enhance the detail features. With the powerful representation ability, only simple fusion strategies are utilized to achieve better fusion performance. The superiority of our method compared with other state-of-the-art methods is demonstrated in qualitative and quantitative experiments on multiple image fusion benchmarks.

show abstract

Section: Experimental Results and Analysismentioning

confidence: 99%

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion

Li,

Xiao,

Cheng

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…2 H W m \left.\right)$ by calculating the axial attention on the H and W axes, respectively. [ 17 ] The axial attention in the widthaxis (high‐axis) is shown in Figure 4b.

F_{v i s}^{1}

is processed by axial attention mechanism to obtain

F_{A X I}

\in �?^{C \times H \times W}

.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…This reduces the complexity from OðHWm 2 Þ to Oð2HWmÞ by calculating the axial attention on the H and W axes, respectively. [17] The axial attention in the widthaxis (high-axis) is shown in Figure 4b. F 1 vis is processed by axial attention mechanism to obtain F AXI ∈ ℝ CÂHÂW .…”

Section: Axial-attention Modulementioning

confidence: 99%

See 1 more Smart Citation

EMA‐GAN: A Generative Adversarial Network for Infrared and Visible Image Fusion with Multiscale Attention Network and Expectation Maximization Algorithm

Xi,

Jin,

Jiang

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The purpose of the infrared and visible image fusion is to generate a fused image with rich information. Although most fusion methods can achieve good performance, there are still shortcomings in extracting feature information from source images, which make it difficult to balance the thermal radiation region information and texture detail information in the fused image. To address the above issues, an expectation maximization (EM) learning framework based on adversarial generative networks (GAN) for infrared and visible image fusion is proposed. The EM algorithm (EMA) can obtain maximum likelihood estimation for problems with potential variables, which is helpful in solving the problem of lack of labels in infrared and visible image fusion. The axial‐corner attention mechanism is designed to capture long‐range semantic information and texture information of the visible image. The multifrequency attention mechanism digs the relationships between features at different scales to highlight target information of infrared images in the fused result. Meanwhile, two discriminators are used to balance two different features, and a new loss function is designed to maximize the likelihood estimate of the data with soft class label assignments, which is obtained from the expectation network. Extensive experiments demonstrate the superiority of EMA‐GAN over the state‐of‐the‐art.

show abstract

“…All mentioned MARL algorithms have been implemented in environments such as StarCraft II [32] or other benchmark problems with a single common reward signal. On the other hand, AIIR-MIX in [33] is an extended QMIX in which agents consider self-generated internal rewards based on state information in addition to external rewards. This corresponds to the goal of influencing the behavior of individual agents or a group of agents in a multi-agent system, which COMA and Consensus-oriented Strategy (CoS) [34] also pursue.…”

Section: Multi Agent Reinforcement Learningmentioning

confidence: 99%

Multi-Agent Active Perception Based on Reinforcement Learning and POMDP

Selimović,

Peti,

Bogdan

2024

IEEE Access

View full text Add to dashboard Cite

In this article, we address a form of active perception characterized by curiosity-driven, openended exploration with intrinsic motivation, carried out by a group of agents. The multiple agents and a large number of possible actions are the main motivation for incorporating Multi-Agent Reinforcement Learning used to train a neural network in order to derive agent's policy. Partially Observable Markov Decision Process framework is used to accommodate the inaccuracy of sensors and probabilistic nature of agent's actions. The proposed method incorporates a consensus that derives the common belief vector, thus allowing each agent to make its decisions based on information acquired by all agents involved in the process of active perception. A well-known benchmark problem with a decentralized tiger scenario was used to demonstrate the possibility of the method to generate agents with different perceptual characteristics by simply changing the agents' reward function related to their intrinsic motivation. The main validation of the proposed approach was performed by using an example of multi-agent search mission. Final results are presented and discussed, and possible avenues for progress on open problems are identified.

show abstract

AEFusion: A multi-scale fusion network combining Axial attention and Entropy feature Aggregation for infrared and visible images

Cited by 15 publications

References 52 publications

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion

EMA‐GAN: A Generative Adversarial Network for Infrared and Visible Image Fusion with Multiscale Attention Network and Expectation Maximization Algorithm

Multi-Agent Active Perception Based on Reinforcement Learning and POMDP

Contact Info

Product

Resources

About