ContriSci: A BERT-Based Multitasking Deep Neural Architecture to Identify Contribution Statements from Research Papers

Gupta, Komal; Ahmad, Ammaar; Ghosal, Tirthankar; Ekbal, Asif

doi:10.1007/978-3-030-91669-5_34

Cited by 4 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Banerjee et al [5] apply sequential transfer learning from the medical to the computer science domain for discourse classification, however, only for two domains and on abstracts, whereas Spangher et al [66] explore this task on news articles with multi-task learning using multiple datasets. Gupta et al [29] utilise a multi-task learning with two scaffold tasks to detect contribution sentences in full papers, however, only in one domain and with limited sentence context. Several approaches also exist to train multiple tasks jointly: Luan et al [46] train a model on three tasks (coreference resolution, entity and relation extraction) using one dataset of research papers.…”

Section: Transfer Learningmentioning

confidence: 99%

“…First, although some approaches propose transfer learning for the scientific domain [5,10,29,53], the field lacks a comprehensive empirical study on transfer learning across different scientific domains for sequential sentence classification. Transfer learning enables the combination of knowledge from multiple datasets to improve classification performance and thus to reduce annotation costs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cross-domain multi-task learning for sequential sentence classification in research papers

Brack

Hoppe

Buschermöhle

et al. 2022

Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries

View full text Add to dashboard Cite

Sequential sentence classification deals with the categorisation of sentences based on their content and context. Applied to scientific texts, it enables the automatic structuring of research papers and the improvement of academic search engines. However, previous work has not investigated the potential of transfer learning for sentence classification across different scientific domains and the issue of different text structure of full papers and abstracts. In this paper, we derive seven related research questions and present several contributions to address them: First, we suggest a novel uniform deep learning architecture and multi-task learning for cross-domain sequential sentence classification in scientific texts. Second, we tailor two common transfer learning methods, sequential transfer learning and multi-task learning, to deal with the challenges of the given task. Semantic relatedness of tasks is a prerequisite for successful transfer learning of neural models. Consequently, our third contribution is an approach to semi-automatically identify semantically related classes from different annotation schemes and we present an analysis of four annotation schemes. Comprehensive experimental results indicate that models, which are trained on datasets from different scientific domains, benefit from one another when using the proposed multi-task learning architecture. We also report comparisons with several state-of-the-art approaches. Our approach outperforms the state of the art on full paper datasets significantly while being on par for datasets consisting of abstracts.

show abstract