Query-driven Generative Network for Document Information Extraction in the Wild

Cao, Haoyu; Li, Xin; Ma, Jiefeng; Jiang, Deqiang; Guo, Antai; Hu, Yiqing; Líu, Hao; Liu, Yinsong; Ren, Bo

doi:10.1145/3503161.3547877

Cited by 11 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The task of visually rich document understanding (VRDU), which involves extracting information from VRDs [2,18], requires models that can handle various types of documents, such as voice, receipts, forms, emails, and advertisements, and various types of information, including rich visuals, large amounts of text, and complex document layouts [27,17,25]. Recently, fine-tuning based on pre-trained visual document understanding models has yielded impressive results in extracting information from VRDs [40,12,21,22,14,20], suggesting that the use of large-scale, unlabeled training documents in pre-training document understanding models can benefit information extraction from VRDs.…”

Section: Introductionmentioning

confidence: 99%

ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction

He¹,

Wang²,

Hu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large language models (LLMs), such as GPT-3 and ChatGPT, have demonstrated remarkable results in various natural language processing (NLP) tasks with incontext learning, which involves inference based on a few demonstration examples. Despite their successes in NLP tasks, no investigation has been conducted to assess the ability of LLMs to perform document information extraction (DIE) using in-context learning. Applying LLMs to DIE poses two challenges: the modality and task gap. To this end, we propose a simple but effective in-context learning framework called ICL-D3IE, which enables LLMs to perform DIE with different types of demonstration examples. Specifically, we extract the most difficult and distinct segments from hard training documents as hard demonstrations for benefiting all test instances. We design demonstrations describing relationships that enable LLMs to understand positional relationships. We introduce formatting demonstrations for easy answer extraction. Additionally, the framework improves diverse demonstrations by updating them iteratively. Our experiments on three widely used benchmark datasets demonstrate that the ICL-D3IE framework enables GPT-3/ChatGPT to achieve superior performance when compared to previous pretrained methods fine-tuned with full training in both the in-distribution (ID) setting and in the out-of-distribution (OOD) setting.

show abstract