Uncertainty-based Fusion Netwok for Automatic Skin Lesion Diagnosis

Chen, Shubai; Yang, Xiaodong; Chen, Yiqiang; Yu, Hanchao; Cai, Hong

doi:10.1109/bibm55620.2022.9994932

Cited by 4 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By introducing dynamics, they can integrate reliable cues from multi-spectral data for around-the-clock applications (e.g., pedestrian detection in security surveillance and autonomous driving). Combining with additional dynamic mechanism (e.g., a simple weighting strategy or Dempster-Shafer Evidence Theory (Shafer, 1976)), recent uncertaintybased multimodal fusion methods show remarkable advantages in various tasks, including clustering (Geng et al, 2021), classification (Han et al, 2021;Tellamekala et al, 2022;Subedar et al, 2019;Chen et al, 2022a), regression (Ma et al, 2021), object detection (Zhang et al, 2019;Li et al, 2022b) and semantic segmentation (Tian et al, 2020;Chang et al, 2022).…”

Section: Multimodal Fusionmentioning

confidence: 99%

Predicting Social Events with Multimodal Fusion of Spatial and Temporal Dynamic Graph Representations

Zhang

Tan³

et al. 2022

Big Data

View full text Add to dashboard Cite

The inherent challenge of multimodal fusion is to precisely capture the cross-modal correlation and flexibly conduct cross-modal interaction. To fully release the value of each modality and mitigate the influence of low-quality multimodal data, dynamic multimodal fusion emerges as a promising learning paradigm. Despite its widespread use, theoretical justifications in this field are still notably lacking. Can we design a provably robust multimodal fusion method? This paper provides theoretical understandings to answer this question under a most popular multimodal fusion framework from the generalization perspective. We proceed to reveal that several uncertainty estimation solutions are naturally available to achieve robust multimodal fusion. Then a novel multimodal fusion framework termed Quality-aware Multimodal Fusion (QMF) is proposed, which can improve the performance in terms of classification accuracy and model robustness. Extensive experimental results on multiple benchmarks can support our findings.

show abstract