Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models

He, Tianxing; Li, Jun; Cho, Kyunghyun; Ott, Myle; Liu, Bing; Glass, James; Peng, Fuchun

doi:10.18653/v1/2021.eacl-main.95

Cited by 27 publications

(21 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While fine-tuning pre-trained representations usually provides strong empirical performance (Wang et al, 2018;Talmor et al, 2020), how fine-tuning manage to do so has remained an open question. Moreover, the instability (Mosbach et al, 2020a;Dodge et al, 2020;Zhang et al, 2020) and forgetting problems He et al, 2021) make it harder to analyze fine-tuned representations. Despite these difficulties, previous work (Merchant et al, 2020;Mosbach et al, 2020b;Hao et al, 2020) draw valuable conclusions about fine-tuning.…”

Section: Related Workmentioning

confidence: 99%

A Closer Look at How Fine-tuning Changes BERT

Zhou¹,

Srikumar²

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Given the prevalence of pre-trained contextualized representations in today's NLP, there have been many efforts to understand what information they contain, and why they seem to be universally successful. The most common approach to use these representations involves fine-tuning them for an end task. Yet, how fine-tuning changes the underlying embedding space is less studied. In this work, we study the English BERT family and use two probing techniques to analyze how fine-tuning changes the space. We hypothesize that fine-tuning affects classification performance by increasing the distances between examples associated with different labels. We confirm this hypothesis with carefully designed experiments on five different NLP tasks. Via these experiments, we also discover an exception to the prevailing wisdom that "fine-tuning always improves performance". Finally, by comparing the representations before and after fine-tuning, we discover that fine-tuning does not introduce arbitrary changes to representations; instead, it adjusts the representations to downstream tasks while largely preserving the original spatial structure of the data points.

show abstract

Section: Related Workmentioning

confidence: 99%

A Closer Look at How Fine-tuning Changes BERT

Zhou¹,

Srikumar²

2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

show abstract

“…While fine-tuning pre-trained representations usually provides strong empirical performance (Wang et al, 2018;Talmor et al, 2020), how fine-tuning changes the representation to do so has remained an open question. Moreover, the instability (Mosbach et al, 2020a;Dodge et al, 2020; and forgetting problems He et al, 2021) of fine-tuning make it harder to analyze fine-tuned representations. With these difficulties, previous work (Merchant et al, 2020;Mosbach et al, 2020b;Hao et al, 2020) draw valuable conclusions about fine-tuning.…”

Section: Related Workmentioning

confidence: 99%

A Closer Look at How Fine-tuning Changes BERT

Zhou¹,

Srikumar²

2021

Preprint

View full text Add to dashboard Cite

Given the prevalence of pre-trained contextualized representations in today's NLP, there have been several efforts to understand what information such representations contain. A common strategy to use such representations is to fine-tune them for an end task. However, how fine-tuning for a task changes the underlying space is less studied. In this work, we study the English BERT family and use two probing techniques to analyze how fine-tuning changes the space. Our experiments reveal that fine-tuning improves performance because it pushes points associated with a label away from other labels. By comparing the representations before and after fine-tuning, we also discover that fine-tuning does not change the representations arbitrarily; instead, it adjusts the representations to downstream tasks while preserving the original structure. Finally, using carefully constructed experiments, we show that fine-tuning can encode training sets in a representation, suggesting an overfitting problem of a new kind.

show abstract

“…Such models can respond well even in challenging dialogue tasks Adiwardana et al, 2020;Huang et al, 2020). Due to the hardware and data requirements of such models, fine-tuning pre-trained models is a popular approach for obtaining well-performing language generation models (Howard and Ruder, 2018;Wolf et al, 2019a;He et al, 2021). Lack of consistency is one of the major issues in neural dialogue generation, which has been tackled by methods such as including persona or situation description to improve the consistency between generated sentences across multiple turns of dialogue.…”

Section: Related Workmentioning

confidence: 99%

Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation

Nyberg,

Manuvinakurike,

Paetzel-Prüsmann

2021

Preprint

View full text Add to dashboard Cite

Human ratings are one of the most prevalent methods to evaluate the performance of natural language processing algorithms. Similarly, it is common to measure the quality of sentences generated by a natural language generation model using human raters. In this paper, we argue for exploring the use of subjective evaluations within the process of training language generation models in a multi-task learning setting. As a case study, we use a crowdauthored dialogue corpus to fine-tune six different language generation models. Two of these models incorporate multi-task learning and use subjective ratings of lines as part of an explicit learning goal. A human evaluation of the generated dialogue lines reveals that utterances generated by the multi-tasking models were subjectively rated as the most typical, most moving the conversation forward, and least offensive. Based on these promising first results, we discuss future research directions for incorporating subjective human evaluations into language model training and to hence keep the human user in the loop during the development process.

show abstract

Analyzing the Forgetting Problem in Pretrain-Finetuning of Open-domain Dialogue Response Models

Cited by 27 publications

References 22 publications

A Closer Look at How Fine-tuning Changes BERT

A Closer Look at How Fine-tuning Changes BERT

A Closer Look at How Fine-tuning Changes BERT

Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation

Contact Info

Product

Resources

About