Syntactic priming in the L2 neural language model

Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models

“…Structural priming effects have been observed in humans both within a given language (Bock, 1986;Ferreira and Bock, 2006;Pickering and Ferreira, 2008;Dell and Ferreira, 2016;Mahowald et al, 2016;Branigan and Pickering, 2017) and crosslingually (Loebell and Bock, 2003;Hartsuiker et al, 2004;Schoonbaert et al, 2007;Shin and Christianson, 2009;Bernolet et al, 2013;van Gompel and Arai, 2018;Kotzochampou and Chondrogianni, 2022). In language models, previous work has demonstrated structural priming effects in English (Prasad et al, 2019;Sinclair et al, 2022;Choi and Park, 2022), and initial results have found priming effects between English and Dutch in LSTM language models (Frank, 2021). As these studies argue, the structural priming approach avoids several possible assumptions and confounds found in previous work investigating abstraction in grammatical learning.…”

“…We apply this same logic to language models in the present study. While several previous studies have explored structural priming in language models (Prasad et al, 2019;Sinclair et al, 2022;Frank, 2021;Choi and Park, 2022), to the best of our knowledge, this is the first to look at crosslingual structural priming in Transformer language models. We replicate eight human psycholinguistic studies, investigating structural priming in English, Dutch (Schoonbaert et al, 2007;Bernolet et al, 2013), Spanish (Hartsuiker et al, 2004), German (Loebell and Bock, 2003), Greek (Kotzochampou and Chondrogianni, 2022), Polish , and Mandarin (Cai et al, 2012).…”