DuReadervis: A : A Chinese Dataset for Open-domain Document Visual Question Answering

Qi, Le; Lv, Shangwen; Li, Hongyu; Liu, Jing; She, Qiaoqiao; Wang, Hua; Wang, Haifeng; Liu, Ting

doi:10.18653/v1/2022.findings-acl.105

Cited by 8 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors were successful in creating a novel dataset and fundamental English methodologies. Inspired by that success, various further studies have been created and implemented in a variety of languages [6] including Chinese [7], Japanese [8], and Vietnamese [9].…”

Section: Background and Related Workmentioning

confidence: 99%

Evjvqa Challenge: Multilingual Visual Question Answering

Luu-Thuy Nguyen,

Nghia Hieu Nguyen,

T.D. Vo

et al. 2023

JCC

View full text Add to dashboard Cite

Visual Question Answering (VQA) is a challenging task of natural language processing (NLP) and computer vision (CV), attracting significant attention from researchers. English is a resource-rich language that has witnessed various developments in datasets and models for visual question answering. Visual question answering in other languages also would be developed for resources and models. In addition, there is no multilingual dataset targeting the visual content of a particular country with its own objects and cultural characteristics. To address the weakness, we provide the research community with a benchmark dataset named EVJVQA, including 33,000+ pairs of question-answer over three languages: Vietnamese, English, and Japanese, on approximately 5,000 images taken from Vietnam for evaluating multilingual VQA systems or models. EVJVQA is used as a benchmark dataset for the challenge of multilingual visual question answering at the 9th Workshop on Vietnamese Language and Speech Processing (VLSP 2022). This task attracted 62 participant teams from various universities and organizations. In this article, we present details of the organization of the challenge, an overview of the methods employed by shared-task participants, and the results. The highest performances are 0.4392 in F1-score and 0.4009 in BLUE on the private test set. The multilingual QA systems proposed by the top 2 teams use ViT for the pre-trained vision model and mT5 for the pre-trained language model, a powerful pre-trained language model based on the transformer architecture. EVJVQA is a challenging dataset that motivates NLP and CV researchers to further explore the multilingual models or systems for visual question answering systems.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Evjvqa Challenge: Multilingual Visual Question Answering

Luu-Thuy Nguyen,

Nghia Hieu Nguyen,

T.D. Vo

et al. 2023

JCC

View full text Add to dashboard Cite

show abstract

“…There are however few text-only QA datasets that contain such answer types [69,46,18]. Other datasets mainly related to our work are rather domain-specific like [112,87,56,86,73]. We give a detailed comparison of most related document VQA datasets in Table 1 highlighting the major contributions.…”

Section: Related Workmentioning

confidence: 99%

Document Understanding Dataset and Evaluation (DUDE)

Landeghem,

Powalski,

Tito

et al. 2023

2023 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We call on the Document AI (DocAI) community to reevaluate current methodologies and embrace the challenge of creating more practically-oriented benchmarks. Document Understanding Dataset and Evaluation (DUDE) seeks to remediate the halted research progress in understanding visually-rich documents (VRDs). We present a new dataset 1 with novelties related to types of questions, answers, and document layouts based on multi-industry, multi-domain, and multi-page VRDs of various origins, and dates. Moreover, we are pushing the boundaries of current methods by creating multi-task and multi-domain evaluation setups that more accurately simulate real-world situations where powerful generalization and adaptation under low-resource settings are desired. DUDE aims to set a new standard as a more practical, long-standing benchmark for the community, and we hope that it will lead to future extensions and contributions that address real-world challenges. Finally, our work illustrates the importance of finding more efficient ways to model language, images, and layout in DocAI.1 huggingface.co/datasets/jordyvl/DUDE_loader

show abstract

“…DocVQA (Mathew, Karatzas, and Jawahar 2021) performs question answering on industry documents. VisualMRC (Tanaka, Nishida, and Yoshida 2021), WebSRC (Chen et al 2021), WebQA (Chang et al 2022) and DuReader vis (Qi et al 2022) require comprehension on web pages. InfographicVQA (Mathew et al 2022) focuses on arithmetic reasoning over infographics.…”

Section: Multimodal Question Answeringmentioning

confidence: 99%

Commercialization of state-owned broadcast networks in China

Yan¹

View full text Add to dashboard Cite

Visual contents, such as illustrations and images, play a big role in product manual understanding. Existing Product Manual Question Answering (PMQA) datasets tend to ignore visual contents and only retain textual parts. In this work, to emphasize the importance of multimodal contents, we propose a Multimodal Product Manual Question Answering (MPMQA) task. For each question, MPMQA requires the model not only to process multimodal contents but also to provide multimodal answers. To support MPMQA, a large-scale dataset PM209 is constructed with human annotations, which contains 209 product manuals from 27 well-known consumer electronic brands. Human annotations include 6 types of semantic regions for manual contents and 22,021 pairs of question and answer. Especially, each answer consists of a textual sentence and related visual regions from manuals. Taking into account the length of product manuals and the fact that a question is always related to a small number of pages, MP-MQA can be naturally split into two subtasks: retrieving most related pages and then generating multimodal answers. We further propose a unified model that can perform these two subtasks all together and achieve comparable performance with multiple task-specific models. The PM209 dataset is available at https://github.com/AIM3-RUC/MPMQA.

show abstract

DuReadervis: A : A Chinese Dataset for Open-domain Document Visual Question Answering

Cited by 8 publications

References 16 publications

Evjvqa Challenge: Multilingual Visual Question Answering

Evjvqa Challenge: Multilingual Visual Question Answering

Document Understanding Dataset and Evaluation (DUDE)

Commercialization of state-owned broadcast networks in China

Contact Info

Product

Resources

About