Gesture in the eye of the beholder: An eye-tracking study on factors determining the attention for gestures produced by people with aphasia

Abstract

Co-speech hand gestures are a ubiquitous form of nonverbal communication, which can express additional information that is not present in speech. Hand gestures may become more relevant when speech production is impaired as in patients with post-stroke aphasia. In fact, patients with aphasia produce more gestures than control speakers. Further, their gestures seem to be more relevant for the understanding of their communication. In the present study, we addressed the question whether the gestures produced by speakers with aphasia catch the attention of their addressees. Healthy volunteers (observers) watched short video clips while their eye movements were recorded. These video clips featured speakers with aphasia and control speakers describing two different scenarios (buying a sweater or having witnessed an accident). Our results show that hand gestures produced by speakers with aphasia are on average longer attended than gestures produced by control speakers. This effect is significant even when we control for the longer duration of the gestural movements in speakers with aphasia. Further, the amount of information in speech was correlated with gesture attention: gestures produced by speakers with less informative speech were attended more frequently. In conclusion, our results highlight two main points. First, overt attention for co-speech hand gesture increases with their communicative relevance. Second, these findings have clinical implications because they show that the extra effort that speakers with aphasia put into gesture is worthwhile, as interlocutors seem to notice their gestures.