Global-to-Local Neural Networks for Document-Level Relation Extraction

Wang, Difeng; Hu, Wei; Cao, Ermei; Sun, Weijian

doi:10.48550/arxiv.2009.10359

Cited by 5 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…and then perform inferences through GCN (Kipf and Welling, 2016) on the graph. We include GEDA , LSR (Nan et al, 2020), GLRE (Wang et al, 2020a), GAIN (Zeng et al, 2020), and HeterGSAN (Xu et al, 2021b) for comparison.…”

Section: Results On the Docred Datasetmentioning

confidence: 99%

“…On the CDR and GDA data sets, we compared BRAN (Verga et al, 2018), EoG (Christopoulou et al, 2019), LSR (Nan et al, 2020), DHG (Zhang et al, 2020c), GLRE (Wang et al, 2020a), ATLOP (Zhou et al, 2021), and DocuNet with our model. The experimental results on two biomedical datasets are shown in Table 2.…”

Section: Results On the Biomedical Datasetsmentioning

confidence: 99%

“…LSR (Nan et al, 2020) 64.8 82.2 DHG (Zhang et al, 2020c) 65.9 83.1 GLRE (Wang et al, 2020a) 68.5 -SciBERT base (Beltagy et al, 2019) 65.1 82.5 ATLOP-SciBERT base (Zhou et al, 2021) 69.4 83.9…”

Section: Ablation Studymentioning

confidence: 99%

“…Specifically, they first constructed a graph structure from the document, and then applied the GCN (Kipf and Welling, 2016;Huang et al, 2017) to the graph to complete logical reasoning. The graph-based method was first introduced by (Quirk and Poon, 2016) and has recently been extended by many works (Christopoulou et al, 2019;Zhang et al, 2020c;Zhou et al, 2020;Wang et al, 2020a;Nan et al, 2020;Zeng et al, 2020;Xu et al, 2021b). proposed the Graph Enhanced Dual Attention network (GEDA) model and used it to characterize the complex interaction between sentences and potential relation instances.…”

Section: Document-level Rementioning

confidence: 99%

“…To extract these complex relationships, most current approaches constructed a document-level graph based on heuristics, structured attention, or dependency structures (Zeng et al, 2020;Nan et al, 2020;Christopoulou et al, 2019;Wang et al, 2020a), and then perform inference with graph convolutional network (GCN) (Guo et al, 2019;Kipf and Welling, 2016) on the graph. Meanwhile, considering the transformer architecture can implicitly model long-distance dependencies, some studies (Tang et al, 2020;Zhou et al, 2021) directly applied pre-trained models rather than explicit graph reasoning .…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction

Zhang¹,

Cheng²

2022

Preprint

View full text Add to dashboard Cite

Document-level relation extraction (RE) aimsto identify relations between two entities in a given document.Compared with its sentence-level counterpart, document-level RE requires complex reasoning. Previous research normally completed reasoning through information propagation on the mention-level or entity-level document-graph, but rarely considered reasoning at the entity-pair-level. In this paper, we propose a novel model, called Densely Connected Criss-Cross Attention Network (Dense-CCNet), for documentlevel RE, which can complete logical reasoning at the entity-pair-level. Specifically, the Dense-CCNet performs entity-pair-level logical reasoning through the Criss-Cross Attention (CCA), which can collect contextual information in horizontal and vertical directions on the entity-pair matrix to enhance the corresponding entity-pair representation. In addition, we densely connect multiple layers of the CCA to simultaneously capture the features of single-hop and multi-hop logical reasoning. We evaluate our Dense-CCNet model on three public document-level RE datasets, DocRED, CDR, and GDA. Experimental results demonstrate that our model achieves state-of-the-art performance on these three datasets.

show abstract

Section: Results On the Docred Datasetmentioning

confidence: 99%

Section: Results On the Biomedical Datasetsmentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 99%

Section: Document-level Rementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction

Zhang¹,

Cheng²

2022

Preprint

View full text Add to dashboard Cite

show abstract

BioGPT: generative pre-trained transformer for biomedical text generation and mining

Luo

Sun

Xia

et al. 2022

Briefings in Bioinformatics

409

159

View full text Add to dashboard Cite

Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main branches of pre-trained language models in the general language domain, i.e. BERT (and its variants) and GPT (and its variants), the first one has been extensively studied in the biomedical domain, such as BioBERT and PubMedBERT. While they have achieved great success on a variety of discriminative downstream biomedical tasks, the lack of generation ability constrains their application scope. In this paper, we propose BioGPT, a domain-specific generative Transformer language model pre-trained on large-scale biomedical literature. We evaluate BioGPT on six biomedical natural language processing tasks and demonstrate that our model outperforms previous models on most tasks. Especially, we get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks, respectively, and 78.2% accuracy on PubMedQA, creating a new record. Our case study on text generation further demonstrates the advantage of BioGPT on biomedical literature to generate fluent descriptions for biomedical terms.

show abstract

A Hierarchical Network for Multimodal Document-Level Relation Extraction

Kong,

Wang,

et al. 2024

AAAI

View full text Add to dashboard Cite

Document-level relation extraction aims to extract entity relations that span across multiple sentences. This task faces two critical issues: long dependency and mention selection. Prior works address the above problems from the textual perspective, however, it is hard to handle these problems solely based on text information. In this paper, we leverage video information to provide additional evidence for understanding long dependencies and offer a wider perspective for identifying relevant mentions, thus giving rise to a new task named Multimodal Document-level Relation Extraction (MDocRE). To tackle this new task, we construct a human-annotated dataset including documents and relevant videos, which, to the best of our knowledge, is the first document-level relation extraction dataset equipped with video clips. We also propose a hierarchical framework to learn interactions between different dependency levels and a textual-guided transformer architecture that incorporates both textual and video modalities. In addition, we utilize a mention gate module to address the mention-selection problem in both modalities. Experiments on our proposed dataset show that 1) incorporating video information greatly improves model performance; 2) our hierarchical framework has state-of-the-art results compared with both unimodal and multimodal baselines; 3) through collaborating with video information, our model better solves the long-dependency and mention-selection problems.

show abstract

Global-to-Local Neural Networks for Document-Level Relation Extraction

Cited by 5 publications

References 35 publications

A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction

A Densely Connected Criss-Cross Attention Network for Document-level Relation Extraction

BioGPT: generative pre-trained transformer for biomedical text generation and mining

A Hierarchical Network for Multimodal Document-Level Relation Extraction

Contact Info

Product

Resources

About