A Tale of Two Modalities: Sign and Speech Influence Each Other in Bimodal Bilinguals

Abstract

Bimodal bilinguals are hearing individuals fluent in a sign and a spoken language. Can the two languages influence each other in such individuals despite differences in the visual (sign) and vocal (speech) modalities of expression? We investigated cross-linguistic influences on bimodal bilinguals’ expression of spatial relations. Unlike spoken languages, sign uses iconic linguistic forms that resemble physical features of objects in a spatial relation and thus expresses specific semantic information. Hearing bimodal bilinguals ( n = 21) fluent in Dutch and Sign Language of the Netherlands and their hearing nonsigning and deaf signing peers ( n = 20 each) described left/right relations between two objects. Bimodal bilinguals expressed more specific information about physical features of objects in speech than nonsigners, showing influence from sign language. They also used fewer iconic signs with specific semantic information than deaf signers, demonstrating influence from speech. Bimodal bilinguals’ speech and signs are shaped by two languages from different modalities.

Development of spatial language and memory

“…In the remaining half of the experimental displays, one non-target picture contained the contrastive spatial configuration (i.e., contrast picture; if the target picture contained Left spatial configuration, contrast picture contained Right spatial configuration or vice versa) and remaining pictures contained spatial configurations other than Left-Right (i.e., Contrast displays) (See Figure 15 for example displays). The rationale for having Contrast displays in addition to Noncontrast displays was to increase the need for informativeness in describing the spatial relation between the objects in the target pictures (i.e., more need for informativeness to describe Contrast than Non-Contrast displays) in a more distinctive way to distinguish it among the other pictures in the display (see Manhardt et al, , 2021 for a similar procedure).…”

“…Even though classifier constructions have been argued to be the most preferred linguistic form to encode spatial relations between the objects in sign languages (e.g., Perniss et al, 2015a;Talmy, 2003), signers can also use other linguistic forms to express spatial relation between the objects such as relational lexemes (akin to spatial nouns or prepositions in spoken languages; Arık, 2003;Sümer, 2015;Manhardt et al, , 2021, tracing the objects' shape and locating them on the signing space (Perniss et al, 2015a), pointing to signing space to indicate the location of objects (Perniss et al, 2015a; as reported in Chapter 3), and lexical verb placements (Newport, 1988) (See Chapter 2 and Chapter 3 for more information and examples regarding these forms). Similar to classifier constructions, all of these linguistic forms encoding spatial relations have analogue mappings between the spatial relations in the real space and the linguistic form in the signing space.…”

“…Informativeness depends on the context in which participants provide a uniquely referring expression that distinguish the target spatial relation between the objects compared to other potential spatial relations. In the present study, participants engaged in a communicative task in which they saw displays with 4 pictures presenting different spatial configurations of the same two objects (see Manhardt et al , 2021 for a similar procedure). Within one display, the only distinguishing feature of the pictures was the spatial configuration between the objects (see Figure 15 for example displays).…”

“…In our study, participants saw displays with two objects in various spatial relations to each other and were asked to describe the target picture (indicated by an arrow) to an addressee (see Chapter 2; as reported in Chapter 4; Manhardt et al, , 2021 for a similar procedure). Later, we tested their recognition memory accuracy of the target pictures in a surprise recognition memory task (see as reported in Chapter 4 for a similar procedure).…”

Sign advantage: Both children and adults’ spatial expressions in sign are more informative than those in speech and gestures combined

“…We defined informativeness in terms of whether a participant's description distinguishes symmetrical Left-Right relations (see Grigoroglou & Papafragou, 2019a for a similar approach in the domain of events). In the present study, participants engaged in a communicative task in which they saw displays with 4 pictures presenting different spatial configurations of the same two objects (see Manhardt et al, 2020;Manhardt, Brouwer, & Özyürek, 2021 for a similar procedure). Within one display, the only distinguishing feature of the pictures was the spatial configuration between the objects (see Figure 4 for examples of displays).…”