LED down the rabbit hole: exploring the potential of global attention for biomedical multi-document summarisation

Abstract

In this paper we report on our submission to the Multidocument Summarisation for Literature Review (MSLR) shared task. Specifically, we adapt PRIMERA (Xiao et al., 2022) to the biomedical domain by placing global attention on important biomedical entities in several ways. We analyse the outputs of the 23 resulting models, and report patterns in the results related to the presence of additional global attention, number of training steps, and the input configuration.* Equal contribution 1 https://github.com/allenai/ mslr-shared-task DatasetWe use the Cochrane dataset as provided in the shared task without any additional data. See Table 5 in Appendix A for dataset statistics. Pre-processingAs the trials are collected automatically from the Cochrane library, they contain redundant metadata 2 Additional results and code for all models is provided at https://github.com/joey234/ PRIMER-pico-attn.