Multi-Modal Weights Sharing and Hierarchical Feature Fusion for RGBD Salient Object Detection

Xiao, Fen; Li, Bin; Peng, Yimu; Cao, Chunhong; Hu, Kai; Gao, Xieping

doi:10.1109/access.2020.2971509

Cited by 4 publications

(2 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed DCNN aims to overcome the limitations of traditional fusion methods and single CNNbased approaches by effectively capturing the complementary information present in multiple modalities and improving the fusion quality. The ensemble learning framework is expected to leverage the diversity and strengths of individual networks to enhance the accuracy, robustness, and interpretability of the fusion process [22].…”

Section: Problem Statementmentioning

confidence: 99%

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

Godla,

Kavitha,

Ashok

et al. 2023

IJACSA

View full text Add to dashboard Cite

Medical image fusion plays a vital role in enhancing the quality and accuracy of diagnostic procedures by integrating complementary information from multiple imaging modalities. In this study, we propose an ensemble learning approach for multi-modal medical image fusion utilizing deep convolutional neural networks (DCNNs) to predict brain tumour. The proposed method aims to exploit the inherent characteristics of different modalities and leverage the power of CNNs for improved fusion results. The Generative Adversarial Network (GAN) strengthens the input images. The ensemble learning framework comprises two main stages. Firstly, a set of DCNN models is trained independently on the respective input modalities, extracting high-level features that capture modalityspecific information. Each DCNN model is fine-tuned to optimize its performance for fusion. Secondly, a fusion module is designed to aggregate the individual modality features and generate a fused image. The fusion module employs a weighted averaging technique to assign appropriate weights to the features based on their relevance and significance. The fused image obtained through this process exhibits enhanced spatial details and improved overall quality compared to the individual modalities. On a diversified dataset made up of multi-modal medical images, thorough tests are carried out to assess the efficacy of the suggested approach. The fusion images exhibit improved visual quality, enhanced feature representation, and better preservation of diagnostic information. The BRATS 2018 dataset, which contains Multi-Modal MRI images and patients' healthcare information were used. The proposed method also demonstrates robustness across different medical imaging modalities, highlighting its versatility and potential for widespread adoption in clinical practice.

show abstract

Section: Problem Statementmentioning

confidence: 99%

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

Godla,

Kavitha,

Ashok

et al. 2023

IJACSA

View full text Add to dashboard Cite

show abstract

“…A number of RGB-D saliency detection methods focus on the fusion of cross-modal information. Xiao et al [54] employed a CNN-based cross-modal transfer learning framework to guide the depth domain feature extraction. Wang et al [55] designed two-streamed convolutional neural networks to extract features and employed a switch map to adaptively fuse the predicted saliency maps.…”

Section: Rgb-d Saliency Detectionmentioning

confidence: 99%

Towards Accurate RGB-D Saliency Detection with Complementary Attention and Adaptive Integration

Liu

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Saliency detection based on the complementary information from RGB images and depth maps has recently gained great popularity. In this paper, we propose Complementary Attention and Adaptive Integration Network (CAAI-Net), a novel RGB-D saliency detection model that integrates complementary attention based feature concentration and adaptive cross-modal feature fusion into a unified framework for accurate saliency detection. Specifically, we propose a context-aware complementary attention (CCA) module, which consists of a feature interaction component, a complementary attention component, and a global-context component. The CCA module first utilizes the feature interaction component to extract rich local context features. The resulting features are then fed into the complementary attention component, which employs the complementary attention generated from adjacent levels to guide the attention at the current layer so that the mutual background disturbances are suppressed and the network focuses more on the areas with salient objects. Finally, we utilize a specially-designed adaptive feature integration (AFI) module, which sufficiently considers the lowquality issue of depth maps, to aggregate the RGB and depth features in an adaptive manner. Extensive experiments on six challenging benchmark datasets demonstrate that CAAI-Net is an effective saliency detection model and outperforms nine state-of-the-art models in terms of four widely-used metrics. In addition, extensive ablation studies confirm the effectiveness of the proposed CCA and AFI modules.

show abstract

RGB-D salient object detection: A survey

et al. 2021

View full text Add to dashboard Cite

Salient object detection, which simulates human visual perception in locating the most significant object(s) in a scene, has been widely applied to various computer vision tasks. Now, the advent of depth sensors means that depth maps can easily be captured; this additional spatial information can boost the performance of salient object detection. Although various RGB-D based salient object detection models with promising performance have been proposed over the past several years, an in-depth understanding of these models and the challenges in this field remains lacking. In this paper, we provide a comprehensive survey of RGB-D based salient object detection models from various perspectives, and review related benchmark datasets in detail. Further, as light fields can also provide depth maps, we review salient object detection models and popular benchmark datasets from this domain too. Moreover, to investigate the ability of existing models to detect salient objects, we have carried out a comprehensive attribute-based evaluation of several representative RGB-D based salient object detection models. Finally, we discuss several challenges and open directions of RGB-D based salient object detection for future research. All collected models, benchmark datasets, datasets constructed for attribute-based evaluation, and related code are publicly available at https://github.com/taozh2017/RGBD-SODsurvey.

show abstract

Multi-Modal Weights Sharing and Hierarchical Feature Fusion for RGBD Salient Object Detection

Cited by 4 publications

References 41 publications

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

An Ensemble Learning Approach for Multi-Modal Medical Image Fusion using Deep Convolutional Neural Networks

Towards Accurate RGB-D Saliency Detection with Complementary Attention and Adaptive Integration

RGB-D salient object detection: A survey

Contact Info

Product

Resources

About