Wide coverage biomedical event extraction using multiple partially overlapping corpora

Abstract

BackgroundBiomedical events are key to understanding physiological processes and disease, and wide coverage extraction is required for comprehensive automatic analysis of statements describing biomedical systems in the literature. In turn, the training and evaluation of extraction methods requires manually annotated corpora. However, as manual annotation is time-consuming and expensive, any single event-annotated corpus can only cover a limited number of semantic types. Although combined use of several such corpora could potentially allow an extraction system to achieve broad semantic coverage, there has been little research into learning from multiple corpora with partially overlapping semantic annotation scopes.ResultsWe propose a method for learning from multiple corpora with partial semantic annotation overlap, and implement this method to improve our existing event extraction system, EventMine. An evaluation using seven event annotated corpora, including 65 event types in total, shows that learning from overlapping corpora can produce a single, corpus-independent, wide coverage extraction system that outperforms systems trained on single corpora and exceeds previously reported results on two established event extraction tasks from the BioNLP Shared Task 2011.ConclusionsThe proposed method allows the training of a wide-coverage, state-of-the-art event extraction system from multiple corpora with partial semantic annotation overlap. The resulting single model makes broad-coverage extraction straightforward in practice by removing the need to either select a subset of compatible corpora or semantic types, or to merge results from several models trained on different individual corpora. Multi-corpus learning also allows annotation efforts to focus on covering additional semantic types, rather than aiming for exhaustive coverage in any single annotation effort, or extending the coverage of semantic types annotated in existing corpora.

Event Extraction from Biomedical Literature

“…In each module, the corresponding model solves a multi-class multi-label classification problem with a one-versus-rest SVM classification scheme [50]. Event-Mine uses a rich set of features for the classification problem [17], including token-based features, neighbouring word features, and features that are generated by GDep [51] and Enju [52] parsers. In contrast to TEES, features generated in EventMine are mapped to a hash table to reduce the memory usage.…”

“…In the 2013 BioNLP-ST competition, EventMine ranked first and second in the PC and CG tasks with F-scores of 52.84% and 52.09%, respectively [48,6]. Similar to the combined TEES + EVEX system [8], EventMine has been used to improve event extraction across multiple corpora with partially overlapping semantic annotations, such as the GE and ID corpora [17], with the aim of reducing the need for manual annotation of existing semantic types in new corpora and improving learning on heterogenous texts [17,53,5].…”

“…To maintain a balance between precision and recall, one could instead pass several token candidates to the next stage, and subsequently eliminate triggerless events to reduce false positives. Such a strategy is employed by Event-Mine, targeting high recall in its four event extraction detection modules [53,17] by weighing positive instances in combination with trigger filtering techniques and hand crafted rules during the edge detecting phase.…”

“…Generalizing multiple features into fewer categories may alleviate the latter problem to a certain extent, yet it may also introduce over-generalized features, which, with enough training instances, may increase false positive rates. Furthermore, while some feature sets may provide good prediction with particular data, others may not [17].…”

“…Additionally, the schema defines attributes associated with events including polarity, negation and speculation that reshape the contextual information anchored with an event. These attributes are vitally important [17], as for example negation reverses the meaning of an event, and negation of a candidate argument prevents the construction of an event with this argument.…”