DocRED: A Large-Scale Document-Level Relation Extraction Dataset

Yao, Yuan; Ye, Deming; Li, Peng; Han, Xu; Lin, Yankai; Liu, Zhenghao; Liu, Zhiyuan; Huang, Lixin; Zhou, Jie; Sun, Maosong

doi:10.48550/arxiv.1906.06127

Cited by 20 publications

(39 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to evaluate the methodology on multiple datasets with similar relations, we chose a set of relations that appear in both the TACRED (Zhang et al, 2017) and DocRED (Yao et al, 2019) datasets with at least 50 development examples 3 .…”

Section: Problem Statement and Setupmentioning

confidence: 99%

“…TACRED (Zhang et al, 2017), a large-scale multi-class relation extraction dataset built over newswire and web text. And Do-cRED (Yao et al, 2019), a dataset for document level RE, and similarly designed for multi-class prediction. Per our setup above, we changed the setting of both datasets to per relation binary classi-fication.…”

Section: Problem Statement and Setupmentioning

confidence: 99%

“…Setup Distant supervision (Mintz et al, 2009) suggests a method to construct a training dataset based on a large external KB of relation triplets. Yao et al (2019) offered a machine annotated version of Do-cRED constructed by aligning Wikipedia pages with Wikidata. The authors took great care in creating this resource: a high-quality NER model trained on in-domain manually annotated data was used to automatically annotate possible relation arguments; a named entity linker was used to merge entities with similar KB ID; and finally, Wikidata was queried in order to label pairs of linked entities.…”

Section: Distant Supervisionmentioning

confidence: 99%

“…For some relations, even when considering only sentences with entities of the relevant types, the ratio between positive and negative examples is highly skewed toward the latter and obtaining a modest amount of positive examples will require a laborious annotation effort (see §3). While manual annotation of large datasets is a viable approach, it typically requires contracting a team of professional annotators (Doddington et al, 2004;Ellis et al, 2015) or crowd workers (Zhang et al, 2017;Yao et al, 2019) and is not well suited for smaller projects or for ad-hoc extraction tasks.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Bootstrapping Relation Extractors using Syntactic Search by Examples

Eyal¹,

Amrami²,

Taub-Tabib³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

View full text Add to dashboard Cite

The advent of neural-networks in NLP brought with it substantial improvements in supervised relation extraction. However, obtaining a sufficient quantity of training data remains a key challenge. In this work we propose a process for bootstrapping training datasets which can be performed quickly by non-NLP-experts. We take advantage of search engines over syntactic-graphs (Such as Shlain et al. ( 2020)) which expose a friendly by-example syntax. We use these to obtain positive examples by searching for sentences that are syntactically similar to user input examples. We apply this technique to relations from TACRED and Do-cRED and show that the resulting models are competitive with models trained on manually annotated data and on data obtained from distant supervision. The models also outperform models trained using NLG data augmentation techniques. Extending the search-based approach with the NLG method further improves the results.

show abstract

Section: Problem Statement and Setupmentioning

confidence: 99%

Section: Problem Statement and Setupmentioning

confidence: 99%

Section: Distant Supervisionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bootstrapping Relation Extractors using Syntactic Search by Examples

Eyal¹,

Amrami²,

Taub-Tabib³

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

View full text Add to dashboard Cite

show abstract

“…Note that in this work we assume a set of labeled training examples are available, i.e., the ground truth annotations contain complementary supporting paragraphs. Recently there was a growing in such datasets (Yang et al, 2018;Yao et al, 2019), due to the increasing interest in model explainability. Also, such supervision signals can also be obtained with distant supervision.…”

Section: Training With Complementary Regularizationmentioning

confidence: 99%

Complementary Evidence Identification in Open-Domain Question Answering

Mou

Chang

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

View full text Add to dashboard Cite

This paper proposes a new problem of complementary evidence identification for opendomain question answering (QA). The problem aims to efficiently find a small set of passages that covers full evidence from multiple aspects as to answer a complex question. To this end, we proposes a method that learns vector representations of passages and models the sufficiency and diversity within the selected set, in addition to the relevance between the question and passages. Our experiments demonstrate that our method considers the dependence within the supporting evidence and significantly improves the accuracy of complementary evidence selection in QA domain.

show abstract