Scientific Discourse Tagging for Evidence Extraction

Abstract

The biomedical scientific literature comprises a crucial, sometimes life-saving, natural language resource whose size is accelerating over time. The information in this resource tends to follow a style of discourse that is intended to provide scientific explanations for various pieces of evidence derived from experimental findings. Studying the rhetorical structure of the narrative discourse could enable more powerful information extraction methods to automatically construct models of scientific argument from full-text papers. In this paper, we apply richly contextualized deep representation learning to the analysis of scientific discourse structures as a clause-tagging task. We improve the current state-of-the-art clauselevel sequence tagging over text clauses for a set of discourse types (e.g. "hypothesis", "result", "implication", etc.) on scientific paragraphs. Our model uses contextualized embeddings, word-to-clause encoder, and clauselevel sequence tagging models and achieves F1 performance of 0.784.

Multitask Learning for Class-Imbalanced Discourse Classification

“…For variations on the classification task, we consider using a Conditional Random Field layer instead of a simple FFNN layer, which has been shown to improve results (Li et al, 2019a). However, we do not see an improvement in this case, possibly because the Bi-LSTM layer prior to classification was already inducing sequential information to be shared.…”

“…We describe each layer in turn. Sentence-Embedding Modeling The architecture we use to model each supervised task in our multitask setup is inspired by previous work in sentencelevel tagging and discourse learning (Choubey et al, 2020;Li et al, 2019a). As shown in Figure 4, we use a transformer model, RoBERTa-base (Liu et al, 2019), to generate sentence embeddings: each sentence in a document is fed sequentially into the same model, and we use the <s> token from each sentence as the sentence-level embedding.…”