Closed-loop reasoning with graph-aware dense interaction for visual dialog

Liu, An-An; Zhang, Guokai; Xu, Ning; Guo, Jun; Jin, Guoqing; Li, Xuanya

doi:10.1007/s00530-022-00947-1

Cited by 2 publications

(1 citation statement)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GQA dataset, developed by Stanford [22], is a recent addition to their collection, specifically designed to facilitate scene comprehension and reasoning tasks [23]. It comprises a comprehensive set of 113,018 real-world images sourced from the Visual Genome dataset, accompanied by their corresponding scene graphs.…”

Section: Related Workmentioning

confidence: 99%

ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamese

Tran,

Phan,

Nguyen

et al. 2023

Preprint

View full text Add to dashboard Cite

In recent years, Visual Question Answering (VQA) has gained significant attention for its diverse applications, including intelligent car assistance, aiding visually impaired individuals, and document image information retrieval using natural language queries. VQA requires effective integration of information from questions and images to generate accurate answers. Neural models for VQA have made remarkable progress on large-scale datasets, with a primary focus on resource-rich languages like English.To address this, we introduce the ViCLEVR dataset, a pioneering collection for evaluating various visual reasoning capabilities in Vietnamese while mitigating biases. The dataset comprises over 26,000 images and 30,000 question-answer pairs (QAs), each question annotated to specify the type of reasoning involved. Leveraging this dataset, we conduct a comprehensive analysis of contemporary visual reasoning systems, offering valuable insights into their strengths and limitations.Furthermore, we present PhoVIT, a comprehensive multimodal fusion that identifies objects in images based on questions. The architecture effectively employs transformers to enable simultaneous reasoning over textual and visual data, merging both modalities at an early model stage. The experimental findings demonstrate that our proposed model achieves state-of-the-art performance across seven evaluation metrics.

show abstract

Section: Related Workmentioning

confidence: 99%