Neural stages of spoken, written, and signed word processing in beginning second language learners

Abstract

We combined magnetoencephalography (MEG) and magnetic resonance imaging (MRI) to examine how sensory modality, language type, and language proficiency interact during two fundamental stages of word processing: (1) an early word encoding stage, and (2) a later supramodal lexico-semantic stage. Adult native English speakers who were learning American Sign Language (ASL) performed a semantic task for spoken and written English words, and ASL signs. During the early time window, written words evoked responses in left ventral occipitotemporal cortex, and spoken words in left superior temporal cortex. Signed words evoked activity in right intraparietal sulcus that was marginally greater than for written words. During the later time window, all three types of words showed significant activity in the classical left fronto-temporal language network, the first demonstration of such activity in individuals with so little second language (L2) instruction in sign. In addition, a dissociation between semantic congruity effects and overall MEG response magnitude for ASL responses suggested shallower and more effortful processing, presumably reflecting novice L2 learning. Consistent with previous research on non-dominant language processing in spoken languages, the L2 ASL learners also showed recruitment of right hemisphere and lateral occipital cortex. These results demonstrate that late lexico-semantic processing utilizes a common substrate, independent of modality, and that proficiency effects in sign language are comparable to those in spoken language.

Gamma Oscillatory Activity Related to Language Prediction

“…It has been shown that people make predictions at the level of the visual/orthographic features of upcoming words whenever it is possible (Wang et al, 2018;Brothers, Swaab, & Traxler, 2015;Molinaro et al, 2013;Kim & Lai, 2012;Dikker & Pylkkanen, 2011;Laszlo & Federmeier, 2011). The induced gamma activity around 100 msec after the onset of the SFW most likely reflects the visual processing of the presented words (Leonard et al, 2013;Hauk et al, 2006). Therefore, the high correlation of gamma activity between the activation and prediction of the expected words in the HC/C condition might be due to predictions of the visual/orthographic characteristics of the lexical items.…”

“…To examine whether the gamma activity is associated with representational-specific preactivations, we correlated the gamma power induced by the SFWs (i.e., activation amplitude) with the gamma power associated with the prediction of the SFWs (i.e., prediction amplitude) across trials before baseline correction. We focused on the early stage of word encoding, such as visual word-form analysis (Leonard et al, 2013;Hauk, Davis, Ford, Pulvermüller, & Marslen-Wilson, 2006), such that the activation was estimated within the first 200 msec. This was done to isolate the brain activity that reflects the initial analysis of the presented words instead of the integration of the words with previous contexts (Kutas & Federmeier, 2011).…”

Cochlear implantation (CI) for prelingual deafness: the relevance of studies of brain organization and the role of first language acquisition in considering outcome success

“…Cortical circuitry for processing the linguistic aspects of sign in such native signers essentially recapitulates that for hearing people who process their first spoken language “by ear” (e.g., Neville et al, 1998 ; Petitto et al, 2000 ; MacSweeney et al, 2002 , 2004 ; see MacSweeney et al, 2008a ). To date, no studies have explored the neural and cognitive correlates of delayed language acquisition in children who are late sign learners, and those few studies with adult late-sign learners point to a very different pattern of cortical activation in late learners than in native learners of the language, with reduced activation in the perisylvian circuitry for language tasks in the late sign learners (see e.g., Ferjan Ramirez et al, 2013 ; Leonard et al, 2013 ).…”

Chinese Writing of Deaf or Hard-of-Hearing Students and Normal-Hearing Peers from Complex Network Approach

“…Normal-hearing (NH) individuals first learn a spoken language, which is followed by reading and writing. In contrast to the auditory-vocal modality of spoken language, the visual-gestural modality of sign languages is often used among deaf or hard-of-hearing (DHH) individuals (Stokoe, 2003; Leonard et al, 2013). Sign language is very different from written and spoken language in that it is composed of signs corresponding to locations and movements along with facial expressions and body gestures (Emmorey, 2001; Lu et al, 2015).…”