Classification Betters Regression in Query-Based Multi-document Summarisation Techniques for Question Answering

Mollá, Diego; Jones, Christopher

doi:10.1007/978-3-030-43887-6_56

Cited by 3 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by these shortcomings, we propose a novel, humancurated test set for QfS on well-formed questions consisting of high-quality 250 instances. RL for Summarization/QfS: The usage of RL in QfS has been limited to extractive summarization only (Mollá and Jones, 2019;Mollá et al, 2020;Chali and Mahmud, 2021;Shapira et al, 2022). Mollá and Jones (2019); Mollá et al (2020); Shapira et al (2022) use RL to train sentence selector models, which select sentences to be incorporated into the summary.…”

Section: Related Workmentioning

confidence: 99%

“…RL for Summarization/QfS: The usage of RL in QfS has been limited to extractive summarization only (Mollá and Jones, 2019;Mollá et al, 2020;Chali and Mahmud, 2021;Shapira et al, 2022). Mollá and Jones (2019); Mollá et al (2020); Shapira et al (2022) use RL to train sentence selector models, which select sentences to be incorporated into the summary. Chali and Mahmud (2021) present a hybrid summarization, where an extractive module selects text from the document, which is then used by the abstractive module to generate an abstractive summary.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning

Nath,

Bhattacharyya,

Khadilkar

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Query-focused Summarization (QfS) deals with systems that generate summaries from document(s) based on a query. Motivated by the insight that Reinforcement Learning (RL) provides a generalization to Supervised Learning (SL) for Natural Language Generation, and thereby performs better (empirically) than SL, we use an RL-based approach for this task of QfS. Additionally, we also resolve the conflict of employing RL in Transformers with Teacher Forcing. We develop multiple Policy Gradient networks, trained on various reward signals: ROUGE, BLEU, and Semantic Similarity, which lead to a 10 -point improvement over the State-of-the-Art approach on the ROUGE-L metric for a benchmark dataset (ELI5). We also show performance of our approach in zero-shot setting for another benchmark dataset (Debate-Pedia) -our approach leads to results comparable to baselines, which were specifically trained on DebatePedia. To aid the RL training, we propose a better semantic similarity reward, enabled by a novel Passage Embedding scheme developed using Cluster Hypothesis. Lastly, we contribute a gold-standard test dataset to further research in QfS and Long-form Question Answering (LfQA).

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning

Nath,

Bhattacharyya,

Khadilkar

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…The "MQ" team, as in past years, focused on ideal answers, approaching the task as query-based summarisation. In some of their systems the retrain their previous classification and regression approaches [28] in the new training dataset. In addition, they also employ reinforcement learning with Proximal Policy Optimization (PPO) [41] and two variants to represent the input features, namely Word2Vec-based and BERT-based embeddings.…”

Section: Systemsmentioning

confidence: 99%

“…Finally, the "sbert" team, also focused on ideal answers. They experimented with different embedding models and multi-task learning in their systems, using parts from previous "MQU " systems for the pre-processing of data and the prediction step based on classification and regression [28]. In particular, they used a Universal Sentence Embedding Model [9] (BioBERT-NLI 16 ) based on a version of BioBERT fine-tuned on the the SNLI [6] and the MultiNLI datasets as in Sentence-BERT [39].…”

Section: Systemsmentioning

confidence: 99%

Overview of BioASQ 2020: The Eighth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Nentidis¹,

Katsimpras²,

Vandorou³

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper, we present an overview of the eighth edition of the BioASQ challenge, which ran as a lab in the Conference and Labs of the Evaluation Forum (CLEF) 2020. BioASQ is a series of challenges aiming at the promotion of systems and methodologies for largescale biomedical semantic indexing and question answering. To this end, shared tasks are organized yearly since 2012, where different teams develop systems that compete on the same demanding benchmark datasets that represent the real information needs of experts in the biomedical domain. This year, the challenge has been extended with the introduction of a new task on medical semantic indexing in Spanish. In total, 34 teams with more than 100 systems participated in the three tasks of the challenge. As in previous years, the results of the evaluation reveal that the top-performing systems managed to outperform the strong baselines, which suggests that state-of-the-art systems keep pushing the frontier of research through continuous improvements.

show abstract

“…The Macquarie University ("MQU ") team focused on ideal answers and approached the task under a classification approach for snippet relevance [33]. Extending their previous work [31,32] the snippets are marked as summary relevant or not, utilizing w2vec embeddings and tf-idf vectors of the question-sentence pairs, showcasing that a classification scheme is more appropriate than a regression one.…”

Section: Task 7bmentioning

confidence: 99%

Results of the seventh edition of the BioASQ Challenge

Nentidis,

Bougiatiotis,

Krithara

et al. 2020

Preprint

View full text Add to dashboard Cite

The results of the seventh edition of the BioASQ challenge are presented in this paper. The aim of the BioASQ challenge is the promotion of systems and methodologies through the organization of a challenge on the tasks of large-scale biomedical semantic indexing and question answering. In total, 30 teams with more than 100 systems participated in the challenge this year. As in previous years, the best systems were able to outperform the strong baselines. This suggests that stateof-the-art systems are continuously improving, pushing the frontier of research.

show abstract

Classification Betters Regression in Query-Based Multi-document Summarisation Techniques for Question Answering

Cited by 3 publications

References 11 publications

Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning

Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning

Overview of BioASQ 2020: The Eighth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Results of the seventh edition of the BioASQ Challenge

Contact Info

Product

Resources

About