FakeCLIP: Multimodal Fake Caption Detection with Mixed Languages for Explainable Visualization

Purwanto, Christian Nathaniel; Santoso, Joan; Lei, Po-Ruey; Yang, Hui-Kuo; Peng, Wen-Chih

doi:10.1109/taai54685.2021.00010

Cited by 1 publication

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Justifications for multimodal misinformation can be grouped in three categories: (i) identifying which part of the claim input is misleading (e.g. specific areas in a visual claim or words in a textual one) (Kou et al, 2020;Purwanto et al, 2021;Lourenço and Paes, 2022); (ii) providing natural language justifications following human fact-checkers (Yao et al, 2022); (iii) selecting and highlighting evidence parts used for verification (Atanasova et al, 2020;Shang et al, 2022). Justifications serve purposes beyond explaining veracity classification, e.g.…”

Section: Task Formulationmentioning

confidence: 99%

Multimodal Automated Fact-Checking: A Survey

Akhtar,

Schlichtkrull,

Guo

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Misinformation is often conveyed in multiple modalities, e.g. a miscaptioned image. Multimodal misinformation is perceived as more credible by humans, and spreads faster than its text-only counterparts. While an increasing body of research investigates automated fact-checking (AFC), previous surveys mostly focus on text. In this survey, we conceptualise a framework for AFC including subtasks unique to multimodal misinformation. Furthermore, we discuss related terms used in different communities and map them to our framework. We focus on four modalities prevalent in real-world fact-checking: text, image, audio, and video. We survey benchmarks and models, and discuss limitations and promising directions for future research.

show abstract