2020
DOI: 10.48550/arxiv.2010.12836
|View full text |Cite
Preprint
|
Sign up to set email alerts
|

Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation

Abstract: Models pretrained with self-supervised objectives on large text corpora achieve state-of-theart performance on text summarization tasks. However, these models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains. In this work, we introduce a general method, called Wik-iTransfer, for fine-tuning pretrained models for summarization in an unsupervised, datasetspecific manner which makes use of characteristics of the target da… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1

Citation Types

0
1
0

Year Published

2021
2021
2021
2021

Publication Types

Select...
1

Relationship

0
1

Authors

Journals

citations
Cited by 1 publication
(1 citation statement)
references
References 20 publications
0
1
0
Order By: Relevance
“…Patents' language hardly resembles general-discourse English (used in pre-training), but the domain adaptation problem has not been studied in detail. Among the previous works, Aghajanyan et al (2021) propose a second multitask pre-training step, Chen et al (2020) studies models cross domain performance and Fabbri et al (2020) evaluates zero and few shot settings; all these works described applications to the patent domain, among the others.…”
Section: Domain Adaptationmentioning
confidence: 99%
“…Patents' language hardly resembles general-discourse English (used in pre-training), but the domain adaptation problem has not been studied in detail. Among the previous works, Aghajanyan et al (2021) propose a second multitask pre-training step, Chen et al (2020) studies models cross domain performance and Fabbri et al (2020) evaluates zero and few shot settings; all these works described applications to the patent domain, among the others.…”
Section: Domain Adaptationmentioning
confidence: 99%