GenIE: Generative Information Extraction

Josifoski, Martin; Cao, Nicola De; Peyrard, Maxime; Petroni, Filippo; West, Robert

doi:10.18653/v1/2022.naacl-main.342

Cited by 31 publications

(45 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been two branches exploring the use of LLMs in few-shot IE tasks with the aid of ICL. The first branch (Ding et al, 2022;Josifoski et al, 2023) views LLMs as an annotator and generates abundant samples with (pseudo) labels via ICL approaches. They then train SLMs using augmented data to achieve superior performance.…”

Section: Icl In Information Extraction Tasksmentioning

confidence: 99%

See 1 more Smart Citation

Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!

Ma¹,

Cao²,

Hong³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) have made remarkable strides in various tasks. However, whether they are competitive few-shot solvers for information extraction (IE) tasks and surpass fine-tuned small Pre-trained Language Models (SLMs) remains an open problem. This paper aims to provide a thorough answer to this problem, and moreover, to explore an approach towards effective and economical IE systems that combine the strengths of LLMs and SLMs. Through extensive experiments on eight datasets across three IE tasks, we show that LLMs are not effective few-shot information extractors in general, given their unsatisfactory performance in most settings and the high latency and budget requirements. However, we demonstrate that LLMs can well complement SLMs and effectively solve hard samples that SLMs struggle with. Building on these findings, we propose an adaptive filterthen-rerank paradigm, in which SLMs act as filters and LLMs act as rerankers. By utilizing LLMs to rerank a small portion of difficult samples identified by SLMs, our preliminary system consistently achieves promising improvements (2.1% F1-gain on average) on various IE tasks, with acceptable cost of time and money.

show abstract

Section: Icl In Information Extraction Tasksmentioning

confidence: 99%

“…Moreover, the number of outputs and the extracted span within each output are not fixed. Standing with Josifoski et al (2023), we believe ICL approaches are not experienced on such task formats.…”

Section: Large Language Modelsmentioning

confidence: 99%

Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!

Ma¹,

Cao²,

Hong³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Similarly, ) used a self-describing mechanism for fewshot NER, which leverages mention describing and entity generation. GenIE (Josifoski et al 2022) uses the transformer model to extract unstructured text relationally through global structural constraint. And LightNER is addresses class transfer by constructing a unified learnable verbalizer of entity categories and tackles domain transfer with a pluggable guidance module.…”

Section: Related Workmentioning

confidence: 99%

Sentence-to-Label Generation Framework for Multi-task Learning of Japanese Sentence Classification and Named Entity Recognition

Chengguang

Zhang

Mori

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Information extraction(IE) is a crucial subfield within natural language processing. However, for the traditionally segmented approach to sentence classification and Named Entity Recognition, the intricate interactions between these individual subtasks remain largely uninvestigated. In this study, we propose an integrative analysis, converging sentence classification with Named Entity Recognition, with the objective to unveil and comprehend the mutual reinforcement effect within these two information extraction subtasks. To achieve this, we introduce a Sentence Classification and Named Entity Recognition Multi-task (SCNM) approach that combines Sentence Classification (SC) and Named Entity Recognition (NER). We develop a Sentence-to-Label Generation (SLG) framework for SCNM and construct a Wikipedia dataset containing both SC and NER. Using a format converter, we unify input formats and employ a generative model to generate SC-labels, NER-labels, and associated text segments. We propose a Constraint Mechanism (CM) to improve generated format accuracy. Our results show SC accuracy increased by 1.13 points and NER by 1.06 points in SCNM compared to standalone tasks, with CM raising format accuracy from 63.61 to 100. The findings indicate mutual reinforcement effects between SC and NER, and integration enhances both tasks' performance. We additionally implemented the SLG framework on single SC task. It yielded superior accuracies compared to the baseline on two distinct Japanese SC datasets. Notably, in the experiment of few-shot learning, SLG framework shows much better performance than fine-tune method. These empirical findings contribute additional evidence to affirm the efficacy of the SLG framework.

show abstract

“…Our experiments and analysis show that Wikidata Parser produces more accurate triples improving in both precision and recall if compared with the state-of-the-art generative information extraction methods [6,19,32].…”

Section: Knowledge Generation and Linking: Wikidata Parsermentioning

confidence: 99%

“…MD-F1 TYPE-F1 EL-F1 RN-F1 REL-P REL-R REL-F1 Approach SOTA IE Pipeline [19] ----43. 30 For both subject and object, we generate the surface form mention, canonical label, type label, relation label.…”

Section: Knowledge Generation and Linking: Wikidata Parsermentioning

confidence: 99%

Knowledge Graph Induction enabling Recommending and Trend Analysis: A Corporate Research Community Use Case

Mihindukulasooriya¹,

Mike²,

Rossiello³

et al. 2022

Preprint

View full text Add to dashboard Cite

A research division plays an important role of driving innovation in an organization. Drawing insights, following trends, keeping abreast of new research, and formulating strategies are increasingly becoming more challenging for both researchers and executives as the amount of information grows in both velocity and volume. In this paper we present a use case of how a corporate research community, IBM Research, utilizes Semantic Web technologies to induce a unified Knowledge Graph from both structured and textual data obtained by integrating various applications used by the community related to research projects, academic papers, datasets, achievements and recognition. In order to make the Knowledge Graph more accessible to application developers, we identified a set of common patterns for exploiting the induced knowledge and exposed them as APIs. Those patterns were born out of user research which identified the most valuable use cases or user pain points to be alleviated. We outline two distinct scenarios: recommendation and analytics for business use. We will discuss these scenarios in detail and provide an empirical evaluation on entity recommendation specifically. The methodology used and the lessons learned from this work can be applied to other organizations facing similar challenges.

show abstract

GenIE: Generative Information Extraction

Cited by 31 publications

References 22 publications

Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!

Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!

Sentence-to-Label Generation Framework for Multi-task Learning of Japanese Sentence Classification and Named Entity Recognition

Knowledge Graph Induction enabling Recommending and Trend Analysis: A Corporate Research Community Use Case

Contact Info

Product

Resources

About