Neural representational similarity between L1 and L2 in spoken and written language processing

Abstract

Despite substantial research on the brain mechanisms of L1 and L2 processing in bilinguals, it is still unknown whether language modality (i.e., visual vs. auditory) plays a role in determining whether L1 and L2 are processed similarly. Therefore, we examined the neural representational similarity in neural networks between L1 and L2 in spoken and written word processing in Korean–English–Chinese trilinguals. Participants performed both visual and auditory rhyming judgments in the three languages: Korean, English, and Chinese. The results showed greater similarity among the three languages in the auditory modality than in the visual modality, suggesting more differentiated networks for written word processing in the three languages than spoken word processing. In addition, there was less similarity between spoken and written word processing in L1 than the L2s, suggesting a more specialized network for each modality in L1 than L2s. Finally, the similarity between the two L2s (i.e., Chinese and English) was greater than that between each L2 and L1 after task performance was regressed out, especially in the visual modality, suggesting that L2s are processed similarly. These findings provide important insights about spoken and written language processing in the bilingual brain.

Neural representation of phonological information during Chinese character reading

“…The left IFG containing the traditional Broca's area is a key region for language comprehension and production (Fadiga et al, 2009 ; Nishitani et al, 2005 ). Much research has found that the left IFG is critical for reading words in both alphabetic and logographic languages (Dong et al, 2020 ; Jana & Gesa, 2019 ; Kim et al, 2020 ; Liu et al, 2021 ; Tan et al, 2005 ; Tham et al, 2005 ; Wu et al, 2012 ). Further research has revealed that the left inferior frontal gyrus is involved in the retrieval, manipulation, or selection of phonological representations (Booth et al, 2006 ; Cao et al, 2009 , 2011 ; Siok et al, 2003 ; Tan et al, 2005 ).…”

“…Results showed that the left MFG defined by three different ways, bilateral FFG, left ITG, and left MTG still showed significant correlations between neural DSM and rating-based phonological DSM (all ps < .05; see Table 3 and Figure S4). Second, to examine the contribution of the left MFG to phonological processing of Chinese characters, we calculated partial correlations after controlling for the nDSM in the left IFG, a region that has been repeatedly reported in word reading (Dong et al, 2020;Jana et al, 2019;Kim et al, 2020;Liu et al, 2021;Tan et al, 2005;Tham et al, 2005;Wu et al, 2012), speech production (Basilakos et al, 2018;Flinker et al, 2015;Long et al, 2016;Mugler et al, 2018), and working memory (Basilakos et al, 2018;Ekert et al, 2021). Results showed that the left MFG defined based on the localizer task (r = .0155, p = .028), Harvard-Oxford atlas (r = .0172, p = .0163), and literature (r = .0210, p = .0053) still showed significant correlations between nDSM and rating-based phonological DSM.…”

“…Phonological processing (i.e., transforming the visual forms of written words into sounds) is one of key components of reading (Coltheart et al, 2001 ; Harm & Seidenberg, 2004 ). Numerous neuroimaging studies have examined the neural basis of phonological processing in reading by using various reading tasks, such as rhyme judgment (Aparicio et al, 2007 ; Booth et al, 2004 ; Dębska et al, 2019 ; Kim et al, 2020 ; Liu et al, 2009 , 2018 ; Mathur et al, 2020 ; Pillay et al, 2014 ; Tan et al, 2003 ; Yen et al, 2019 ), homophone judgment (Chuchu et al, 2018 ; Gitelman et al, 2005 ; Matsuo et al, 2010 ; Siok et al, 2003 ; Tham et al, 2005 ; Yang & Tan, 2019 ), initial consonant judgment (Lee et al, 2012 ; Siok et al, 2003 ), phonological decision (Emmorey et al, 2016 ; Hartwigsen et al, 2010 ; Liebig et al, 2017 ), silent reading (Cheema et al, 2018 ; Cullum et al, 2019 ; Hartwigsen et al, 2010 ; Kumar & Padakannaya, 2019 ; Lu et al, 2021 ; Osipowicz et al, 2011 ; Qu et al, 2019 ; Wan et al, 2017 ), and overt naming tasks (Al et al, 2020 ; Berken et al, 2015 ; Binder et al, 2005 ; Brumberg et al, 2016 ; Dietz et al, 2005 ; Dong et al, 2020 , 2021 ; Graves et al, 2010 ; Mechelli et al, 2007 ). Researchers have observed consistent activation in language‐related brain areas in the left inferior frontal gyrus, left temporoparietal cortex, and left lateral temporal cortex.…”

Cortical encoding during word reading in bilinguals: parallel interdigitated distributed networks support convergent linguistic functioning

“…RSA has been widely used to evaluate congruent patterns across modalities within/between subjects (Deniz, Nunez-Elizalde, Huth, & Gallant, 2019; Fischer-Baum et al, 2017; Kim, Liu, Liu, & Cao, 2020; Wang et al, 2018; L. Zhao et al, 2017). Specifically, in the current study, RSA bridged brain activity patterns and behavioral measurements in response to task stimuli (Kriegeskorte et al, 2008).…”

“…Representational Similarity Analysis. RSA has been widely used to evaluate congruent patterns across modalities within/between subjects 2,26,27,65,66 . Specifically, in the current study, RSA bridged brain activity patterns and behavioral measurements in response to task stimuli 24 .…”