Neural systems supporting linguistic structure, linguistic experience, and symbolic communication in sign language and gesture

Abstract

Sign languages used by deaf communities around the world possess the same structural and organizational properties as spoken languages: In particular, they are richly expressive and also tightly grammatically constrained. They therefore offer the opportunity to investigate the extent to which the neural organization for language is modality independent, as well as to identify ways in which modality influences this organization. The fact that sign languages share the visual-manual modality with a nonlinguistic symbolic communicative system-gesture-further allows us to investigate where the boundaries lie between language and symbolic communication more generally. In the present study, we had three goals: to investigate the neural processing of linguistic structure in American Sign Language (using verbs of motion classifier constructions, which may lie at the boundary between language and gesture); to determine whether we could dissociate the brain systems involved in deriving meaning from symbolic communication (including both language and gesture) from those specifically engaged by linguistically structured content (sign language); and to assess whether sign language experience influences the neural systems used for understanding nonlinguistic gesture. The results demonstrated that even sign language constructions that appear on the surface to be similar to gesture are processed within the left-lateralized frontal-temporal network used for spoken languages-supporting claims that these constructions are linguistically structured. Moreover, although nonsigners engage regions involved in human action perception to process communicative, symbolic gestures, signers instead engage parts of the languageprocessing network-demonstrating an influence of experience on the perception of nonlinguistic stimuli.S ign languages such as American Sign Language (ASL) are natural human languages with linguistic structure. Signed and spoken languages also largely share the same neural substrates, including left hemisphere dominance revealed by brain injury and neuroimaging. At the same time, sign languages provide a unique opportunity to explore the boundaries of what, exactly, "language" is. Speech-accompanying gesture is universal (1), yet such gestures are not language-they do not have a set of structural components or combinatorial rules and cannot be used on their own to reliably convey information. Thus, gesture and sign language are qualitatively different, yet both convey symbolic meaning via the hands. Comparing them can help identify the boundaries between language and nonlinguistic symbolic communication.Despite this apparently clear distinction between sign language and gesture, some researchers have emphasized their apparent similarities. One construction that has been a focus of contention is "classifier constructions" (also called "verbs of motion"). In ASL, a verb of motion (e.g., moving in a circle) will include a root expressing the motion event, morphemes marking the manner and direction of motion (e.g., forward o...

The Cortical Organization of Syntax

“…Research in the last several decades has revealed that sign languages possess the same core properties of language as spoken ones (Stokoe, 1960;Bellugi & Klima, 1976;Klima & Bellugi, 1979;Petitto 1994;Sandler & Lillo-Martin, 2006), as well as similar overall neurobiological organization (Hickok et al, 1996;Neville et al, 1998;Petitto et al, 2000;Macsweeney et al, 2002;Emmorey et al, 2007;Mayberry et al, 2011). In particular, sign languages appear to rely on the same underlying lexical-syntactic and semantic systems as for spoken languages (MacSweeney et al, 2006;Leonard et al, 2012;Newman et al, 2015;Matchin et al, 2017b).…”

Reduced Left Lateralization of Language in Congenitally Blind Individuals

“…A similar network of left-lateralized frontotemporal areas is involved in processing English, Mandarin Chinese, and various sign languages (e.g., Newman, Supalla, Fernandez, Newport, & Bavelier, 2014;Chee et al, 1999;Neville et al, 1997). Moreover, the seeds of typical language processing appear to be present early on in the developing brain.…”

Signs for Developing Reading : Sign Language and Reading Development in Deaf and Hard-of-Hearing Children

“…Brain imaging studies also indicate that when deaf adults process lexical manual gestures, the activity in language related brain regions is greater than it is for hearing individuals (Cardin, Orfanidou, Rönnberg, Capek, Rudner, & Woll, 2013;Li, Xia, Zhao, & Qi, 2014;Newman, Supalla, Fernandez, Newport, & Bavelier, 2015). These findings demonstrate that the processing of manual gestures changes with sign language experience.…”

“…Although an associative account of imitation suggests that motor expertise, like being a sign language user, should provide an advantage regarding imitation of manual gestures, the theory also suggests that representations that are close enough to a target behavior might be sufficient (Heyes, 2016). On the other hand, sign language experience does seem to lead to changes in the neural processing of any type of manual gestures in adults (Newman et al, 2015;but see, e.g., Cardin et al, 2016). This suggests that with development, a difference between signing and non-signing individuals might emerge.…”