Language modality and temporal structure impact processing: Sign and speech have different windows of integration

Rivolta, Chiara Luna; Costello, Brendan; Carreiras, Manuel

doi:10.1016/j.jml.2021.104283

Cited by 3 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5B and Materials and Methods and for details of these stimuli. The lower accuracy rates are typical of those found in LSE lexical recognition tasks ( 27 , 28 ) and reflect greater dialectal variation of a nonstandardized language.)…”

Section: Resultsmentioning

confidence: 69%

Cross-modal and cross-language activation in bilinguals reveals lexical competition even when words or signs are unheard or unseen

Villameriel

Costello

Giezen

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

We exploit the phenomenon of cross-modal, cross-language activation to examine the dynamics of language processing. Previous within-language work showed that seeing a sign coactivates phonologically related signs, just as hearing a spoken word coactivates phonologically related words. In this study, we conducted a series of eye-tracking experiments using the visual world paradigm to investigate the time course of cross-language coactivation in hearing bimodal bilinguals (Spanish–Spanish Sign Language) and unimodal bilinguals (Spanish/Basque). The aim was to gauge whether (and how) seeing a sign could coactivate words and, conversely, how hearing a word could coactivate signs and how such cross-language coactivation patterns differ from within-language coactivation. The results revealed cross-language, cross-modal activation in both directions. Furthermore, comparison with previous findings of within-language lexical coactivation for spoken and signed language showed how the impact of temporal structure changes in different modalities. Spoken word activation follows the temporal structure of that word only when the word itself is heard; for signs, the temporal structure of the sign does not govern the time course of lexical access (location coactivation precedes handshape coactivation)—even when the sign is seen. We provide evidence that, instead, this pattern of activation is motivated by how common in the lexicon the sublexical units of the signs are. These results reveal the interaction between the perceptual properties of the explicit signal and structural linguistic properties. Examining languages across modalities illustrates how this interaction impacts language processing.

show abstract

Section: Resultsmentioning

confidence: 69%

Cross-modal and cross-language activation in bilinguals reveals lexical competition even when words or signs are unheard or unseen

Villameriel

Costello

Giezen

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The effect of temporal degradation on speech perception has been investigated using several techniques, e.g., periodic interruption 1 – 7 , temporal smearing 8 , 9 , desynchronising narrow-band slits 10 , 11 , and temporal reversal 12 , 13 . Local time-reversal realises manipulation of intelligibility from perfect to almost none, depending on segment duration 7 , 11 , 14 – 26 . In locally time-reversed speech, an original speech waveform is divided into short segments, which are then each reversed in time, and connected again.…”

Section: Introductionmentioning

confidence: 99%

The common limitations in auditory temporal processing for Mandarin Chinese and Japanese

Eguchi

Ueda

Remijn

et al. 2022

Sci Rep

View full text Add to dashboard Cite

The present investigation focused on how temporal degradation affected intelligibility in two types of languages, i.e., a tonal language (Mandarin Chinese) and a non-tonal language (Japanese). The temporal resolution of common daily-life sentences spoken by native speakers was systematically degraded with mosaicking (mosaicising), in which the power of original speech in each of regularly spaced time-frequency unit was averaged and temporal fine structure was removed. The results showed very similar patterns of variations in intelligibility for these two languages over a wide range of temporal resolution, implying that temporal degradation crucially affected speech cues other than tonal cues in degraded speech without temporal fine structure. Specifically, the intelligibility of both languages maintained a ceiling up to about the 40-ms segment duration, then the performance gradually declined with increasing segment duration, and reached a floor at about the 150-ms segment duration or longer. The same limitations for the ceiling performance up to 40 ms appeared for the other method of degradation, i.e., local time-reversal, implying that a common temporal processing mechanism was related to the limitations. The general tendency fitted to a dual time-window model of speech processing, in which a short (~ 20–30 ms) and a long (~ 200 ms) time-window run in parallel.

show abstract

Advances in Literary and (Neuro)Linguistics Skills That Can Contribute to Healing

Martines

2024

Educational Innovation to Address Complex Societal Challenges

View full text Add to dashboard Cite

This chapter analyses the influence of bibliotherapy on the perceptions of readers (children, young people, and [older] adults) with special needs on their form of social interaction. By reading and writing, and by explaining to others what they read and write, by talking about plots, by expressing opinions and emotions, feelings and thoughts are articulated, and expressive (linguistic), relational, cognitive, and neurological abilities are improved. From linguistics (theoretical, applied, and cognitive), literary studies, translation/multilingual environments, and pedagogy (literacy, didactics of language, and literature), methodological approaches and interventions that can be useful and effective in bibliotherapy to activate neurocognitive abilities are provided. Furthermore, bibliotherapy is presented as a coherent option for improving the management of situations of confinement, physical, real, voluntary, imposed, or perceived loneliness.

show abstract

Language modality and temporal structure impact processing: Sign and speech have different windows of integration

Cited by 3 publications

References 37 publications

Cross-modal and cross-language activation in bilinguals reveals lexical competition even when words or signs are unheard or unseen

Cross-modal and cross-language activation in bilinguals reveals lexical competition even when words or signs are unheard or unseen

The common limitations in auditory temporal processing for Mandarin Chinese and Japanese

Advances in Literary and (Neuro)Linguistics Skills That Can Contribute to Healing

Contact Info

Product

Resources

About