On the Interplay Between Fine-tuning and Composition in Transformers

Abstract

Pre-trained transformer language models have shown remarkable performance on a variety of NLP tasks. However, recent research has suggested that phrase-level representations in these models reflect heavy influences of lexical content, but lack evidence of sophisticated, compositional phrase information (Yu and Ettinger, 2020). Here we investigate the impact of fine-tuning on the capacity of contextualized embeddings to capture phrase meaning information beyond lexical content. Specifically, we fine-tune models on an adversarial paraphrase classification task with high lexical overlap, and on a sentiment classification task. After fine-tuning, we analyze phrasal representations in controlled settings following prior work. We find that fine-tuning largely fails to benefit compositionality in these representations, though training on sentiment yields a small, localized benefit for certain models. In follow-up analyses, we identify confounding cues in the paraphrase dataset that may explain the lack of composition benefits from that task, and we discuss potential factors underlying the localized benefits from sentiment training.

Towards Equipping Transformer with the Ability of Systematic Compositionality

“…It's important to note that most existing evaluation methods and datasets for systematic compositionality are only suitable for generative-based language models like SCAN (Higgins et al 2018b) and incontext prompting . Instead, our evaluations follow recent works (Yu and Ettinger 2021;Hendrycks et al 2020;Yu and Ettinger 2020), which are designed for discriminative-based language models like BERT and ours. Since each evaluation task differs, we provide the details in the corresponding subsections.…”

“…As suggested in previous studies (Yu and Ettinger 2021;Cartuyvels, Spinks, and Moens 2021;Hendrycks et al 2020;Yu and Ettinger 2020), our evaluation closely revolves around the characteristics of the systematic compositionality to provide detailed insights. Due to the issue of computational resources, we restrict our analysis and comparison to the widely used BERT and tentatively implement and pretrain a multi-layer CAT from scratch following BERT.…”

“…It has led to improved performance in various NLP tasks such as question answering (Bogin et al 2021) and machine translation (Ataman and Federico 2018). In the era of LLMs, however, recent evidence shows that fine-tuning a given pre-trained model on a specific task may not improve its compositionality capability (Yu and Ettinger 2021). Over-parameterized LLMs, like ChatGPT, are still sensitive to the selection of in-context examples , making their compositionality ability fragile.…”

“…However, previous studies have shown that language models struggle with generalizing through composition (Cartuyvels, Spinks, and Moens 2021; Lake and Baroni 2018;Loula, Baroni, and Lake 2018). Even for large language models (LLMs), recent evidence suggests that they still struggle to establish systematic compositionality after fine-tuning on compositionality-aware datasets (Yu and Ettinger 2021) or prompting with in-context examples . The challenge lies in the fact that the semantics of a group of primitives vary depending on their meanings and how they are combined.…”

How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy

“…Following the research line in understanding the reasons behind the outstanding performance of pre-trained language models and their capabilities, most recent investigations on fine-tuning have been done through probing tasks and by evaluating the encoded linguistic knowledge (Merchant et al, 2020;Mosbach et al, 2020;Talmor et al, 2020;Yu and Ettinger, 2021). These studies demonstrate that most changes in fine-tuning are applied to the upper layers, such that those layers encode taskspecific knowledge, while lower layers are responsible for the core linguistic phenomenon (Durrani et al, 2021).…”