Inconsistency Matters: A Knowledge-guided Dual-inconsistency Network for Multi-modal Rumor Detection

Sun, Mengzhu; Zhang, Xi; Ma, Jianqiang; Liu, Yazheng

doi:10.18653/v1/2021.findings-emnlp.122

Cited by 12 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The issue of model inconsistency has been recognized in different domains. For example, recent works utilize the inconsistency between image and text to detect fake news (Xiong et al 2023;Sun et al 2023). The modal inconsistency issue has also been investigated in sentiment analysis using text and images.…”

Section: Related Workmentioning

confidence: 99%

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

Yao,

Yin,

Cheung

et al. 2024

AAAI

View full text Add to dashboard Cite

The combination of electronic health records (EHR) and medical images is crucial for clinicians in making diagnoses and forecasting prognoses. Strategically fusing these two data modalities has great potential to improve the accuracy of machine learning models in clinical prediction tasks. However, the asynchronous and complementary nature of EHR and medical images presents unique challenges. Missing modalities due to clinical and administrative factors are inevitable in practice, and the significance of each data modality varies depending on the patient and the prediction target, resulting in inconsistent predictions and suboptimal model performance. To address these challenges, we propose DrFuse to achieve effective clinical multi-modal fusion. It tackles the missing modality issue by disentangling the features shared across modalities and those unique within each modality. Furthermore, we address the modal inconsistency issue via a disease-wise attention layer that produces the patient- and disease-wise weighting for each modality to make the final prediction. We validate the proposed method using real-world large-scale datasets, MIMIC-IV and MIMIC-CXR. Experimental results show that the proposed method significantly outperforms the state-of-the-art models.

show abstract

Section: Related Workmentioning

confidence: 99%

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

Yao,

Yin,

Cheung

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Xue et al (2021) mapped text features and visual semantic features to the same semantic space to obtain cross-modal feature representations, and considers the consistency between them. Sun et al (2023) designed a dual-inconsistency network to simultaneously detect cross-modal inconsistency and content-knowledge inconsistency.…”

Section: Related Workmentioning

confidence: 99%

Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformation

Yuan,

Guo,

Qiu

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Mis-and disinformation online have become a major societal problem as major sources of online harms of different kinds. One common form of mis-and disinformation is outof-context (OOC) information, where different pieces of information are falsely associated, e.g., a real image combined with a false textual caption or a misleading textual description. Although some past studies have attempted to defend against OOC mis-and disinformation through external evidence, they tend to disregard the role of different pieces of evidence with different stances. Motivated by the intuition that the stance of evidence represents a bias towards different detection results, we propose a stance extraction network (SEN) that can extract the stances of different pieces of multi-modal evidence in a unified framework. Moreover, we introduce a support-refutation score calculated based on the co-occurrence relations of named entities into the textual SEN. Extensive experiments on a public large-scale dataset demonstrated that our proposed method outperformed the state-ofthe-art baselines, with the best model achieving a performance gain of 3.2% in accuracy.

show abstract

Combating misinformation in the age of LLMs: Opportunities and challenges

Chen,

Shu

2024

AI Magazine

View full text Add to dashboard Cite

Misinformation such as fake news and rumors is a serious threat for information ecosystems and public trust. The emergence of large language models (LLMs) has great potential to reshape the landscape of combating misinformation. Generally, LLMs can be a double‐edged sword in the fight. On the one hand, LLMs bring promising opportunities for combating misinformation due to their profound world knowledge and strong reasoning abilities. Thus, one emerging question is: can we utilize LLMs to combat misinformation? On the other hand, the critical challenge is that LLMs can be easily leveraged to generate deceptive misinformation at scale. Then, another important question is: how to combat LLM‐generated misinformation? In this paper, we first systematically review the history of combating misinformation before the advent of LLMs. Then we illustrate the current efforts and present an outlook for these two fundamental questions, respectively. The goal of this survey paper is to facilitate the progress of utilizing LLMs for fighting misinformation and call for interdisciplinary efforts from different stakeholders for combating LLM‐generated misinformation.

show abstract

Inconsistency Matters: A Knowledge-guided Dual-inconsistency Network for Multi-modal Rumor Detection

Cited by 12 publications

References 33 publications

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformation

Combating misinformation in the age of LLMs: Opportunities and challenges

Contact Info

Product

Resources

About