Fine-tune Bert for DocRED with Two-step Process

Wang, Hong; Focke, Christfried; Sylvester, Rob; Mishra, Narendra Kumar; Wang, William Yang

doi:10.48550/arxiv.1909.11898

Cited by 42 publications

(42 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Effect of Language Models. To analyze the effect of language models, we conduct the following comparison experiments on different model settings: the two-step Bert-based document-level classifier (Wang et al, 2019) upon the constructed pseudo document, denoted as Bert-Two-Step; and our model (DocDS). The results are shown in Table 3 and Figure 2.…”

Section: Detailed Analysismentioning

confidence: 99%

From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension

Yan,

Han,

Sun

et al. 2020

Preprint

View full text Add to dashboard Cite

Distant supervision (DS) is a promising approach for relation extraction but often suffers from the noisy label problem. Traditional DS methods usually represent an entity pair as a bag of sentences and denoise labels using multi-instance learning techniques. The bag-based paradigm, however, fails to leverage the inter-sentence-level and the entity-level evidence for relation extraction, and their denoising algorithms are often specialized and complicated. In this paper, we propose a new DS paradigm-document-based distant supervision, which models relation extraction as a document-based machine reading comprehension (MRC) task. By re-organizing all sentences about an entity as a document and extracting relations via querying the document with relation-specific questions, the document-based DS paradigm can simultaneously encode and exploit all sentence-level, inter-sentence-level, and entity-level evidence. Furthermore, we design a new loss function-DSLoss (distant supervision loss), which can effectively train MRC models using only document, question, answer tuples, therefore noisy label problem can be inherently resolved. Experiments show that our method achieves new state-of-the-art DS performance.

show abstract

Section: Detailed Analysismentioning

confidence: 99%

From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension

Yan,

Han,

Sun

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…BERT-RE: It uses BERT to encode the document, entities are represented by their average word embedding. A BiLinear layer is applied to predict the relation between entity pairs [13].…”

Section: Comparison Models and Evaluation Metricsmentioning

confidence: 99%

“…BERT-Two-Step: Based on BERT-RE, it models the document-level RE through a two-step process. The first step is to predict whether or not two entities have a relation, the second step is to predict the specific relation [13].…”

Section: Comparison Models and Evaluation Metricsmentioning

confidence: 99%

“…Yao et al [15] proposed a large-scale human-annotated document-level RE dataset, DocRED, and first compute the representations for all entities then predict relations for each entity pair by feeding them into a bilinear function. Wang et al [13] used BERT to encode the document, it also used bilinear layer to predict the relation between entity pairs, but it modelled the document-level RE through a two-step process. Most previous methods used only entity-level information and this is not adequate.…”

Section: Related Workmentioning

confidence: 99%

“…Obviously, most traditional sentence-level RE models often fail to generalize extraction to this situation. To move RE forward from sentence level to document level, many efforts have been made [13,15], but most previous methods used only entity-level information and this is not adequate. Thus, there are still some deep-seated problems unsolved in document-level RE.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HIN: Hierarchical Inference Network for Document-Level Relation Extraction

Tang¹,

Cao²,

Zhang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Document-level RE requires reading, inferring and aggregating over multiple sentences. From our point of view, it is necessary for document-level RE to take advantage of multi-granularity inference information: entity level, sentence level and document level. Thus, how to obtain and aggregate the inference information with different granularity is challenging for document-level RE, which has not been considered by previous work. In this paper, we propose a Hierarchical Inference Network (HIN) to make full use of the abundant information from entity level, sentence level and document level. Translation constraint and bilinear transformation are applied to target entity pair in multiple subspaces to get entity-level inference information. Next, we model the inference between entity-level information and sentence representation to achieve sentence-level inference information. Finally, a hierarchical aggregation approach is adopted to obtain the document-level inference information. In this way, our model can effectively aggregate inference information from these three different granularities. Experimental results show that our method achieves state-of-the-art performance on the large-scale Do-cRED dataset. We also demonstrate that using BERT representations can further substantially boost the performance.

show abstract

Document-level Relation Extraction Based on Heterogeneous Graph Convolutional Network and Local Semantic Fusion

Chen

Xu³

2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Document-level relation extraction is a fundamental task of many downstream applications such as knowledge graph and has gained improvement through document graph and sequence models. These methods always utilize the whole document as an essential global feature while ignoring the discrimination of entity representation. Focusing on local semantic feature, a novel model named GCNLEF based on graph convolutional network is proposed in this paper. In the presented method, a heterogeneous graph containing mention nodes and sentence group nodes is constructed first. Then multi-hop path reasoning is presented to infer the relations between entities. Experimental results on DocRED show that the proposed model can achieve 61.45 F 1 score with 90 epochs, improved by 0.15 F 1 score compared with the state-of-the-art method ATLOP.

show abstract

Fine-tune Bert for DocRED with Two-step Process

Cited by 42 publications

References 15 publications

From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension

From Bag of Sentences to Document: Distantly Supervised Relation Extraction via Machine Reading Comprehension

HIN: Hierarchical Inference Network for Document-Level Relation Extraction

Document-level Relation Extraction Based on Heterogeneous Graph Convolutional Network and Local Semantic Fusion

Contact Info

Product

Resources

About