Multimodal Communication in Aphasia: Perception and Production of Co-speech Gestures During Face-to-Face Conversation

Abstract

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities.

The relationship between speech and gestures in persons with aphasia: Evidence from the Czech perspective

“…PWA Mb1 produced 578 words and 119 communicative gestures, Mw2 produced 580 words and 68 communicative gestures, Mb2 used 750 words and 53 communicative gestures, Mw1 used 733 words and 19 communicative gestures and Wtm produced 818 words and 22 communicative gestures. My results are also concordant with those of Preisig (2018), who found that persons with more severe aphasia (but not with apraxia) produce more meaningful gestures than mildly affected people.…”

“…This fact confirms that gestures can transmit important parts of information, but are also largely dependent on speech. Preisig et al (2018) explored Swiss PWA and the main question of their study (2018, p. 9) concerned the question of how PWA perceive and produce co-speech gestures during face-to-face interaction. They examined the impact of co-speech gestures at the level of perception by the eye-tracking method (using voxel-based lesion-symptom mapping, VLSM) in order to connect the production of co-speech gestures and the localization of brain injury.…”

The Role of Iconic Gestures in Speech Comprehension: An Overview of Various Methodologies

“…Aphasia is an acquired disorder that can affect both language production and comprehension ( Preisig et al, 2018 ). While language and gesture production have been vastly studied, literature on gesture comprehension is quite sparse.…”

“…More recently, several studies investigated co-speech gesture-speech integration in aphasia ( Eggenberger et al, 2016 ; Cocks et al, 2018 ; Preisig et al, 2018 ). Preisig et al (2018) showed that during live conversations, co-speech gestures (of all types) attracted the attention of aphasic patients and were more fixated than abstract gestures. The authors suggested that these patients may benefit from the bimodal information presentation to compensate a verbal deficit ( Preisig et al, 2018 ).…”

“… Preisig et al (2018) showed that during live conversations, co-speech gestures (of all types) attracted the attention of aphasic patients and were more fixated than abstract gestures. The authors suggested that these patients may benefit from the bimodal information presentation to compensate a verbal deficit ( Preisig et al, 2018 ). However, because no task was involved, it is unclear whether patients understood and processed the meaning of these gestures.…”