The origin of human multi-modal communication

Abstract

One reason for the apparent gulf between animal and human communication systems is that the focus has been on the presence or the absence of language as a complex expressive system built on speech. But language normally occurs embedded within an interactional exchange of multi-modal signals. If this larger perspective takes central focus, then it becomes apparent that human communication has a layered structure, where the layers may be plausibly assigned different phylogenetic and evolutionary origins—especially in the light of recent thoughts on the emergence of voluntary breathing and spoken language. This perspective helps us to appreciate the different roles that the different modalities play in human communication, as well as how they function as one integrated system despite their different roles and origins. It also offers possibilities for reconciling the ‘gesture-first hypothesis’ with that of gesture and speech having evolved together, hand in hand—or hand in mouth, rather—as one system

Learned vocal and breathing behavior in an enculturated gorilla

“…Many theories on the origins of language begin with two basic premises concerning the vocal behavior of nonhuman primates, especially the great apes. They assume that (1) apes (and other primates) can exercise only negligible volitional control over the production of sound with their vocal tract, and (2) they are unable to learn novel vocal behaviors beyond their species-typical repertoire (e.g., Arbib et al 2008;Burling 1993;Call and Tomasello 2007;Corballis 2002;Hauser 1996;Levinson and Holler 2014;Pinker 1994;Pollick and De Waal 2007;Premack 2004). These assumptions are often perpetuated through reference to a few notable reports.…”

Social play as joint action: A framework to study the evolution of shared intentionality as an interactional achievement

“…By enabling sophisticated forms of joint action, the human interaction engine paves the way for the emergence of cumulative culture and the development of social institutions. Levinson and Holler (2014) further suggested that the human interaction engine emerged as a step in an increasingly stratified system of communicative competencies that mark the evolution of modern human communication and has potentially evolved around 2 mya with the early forms of Homo.…”

“…are expressions of a uniquely human set of capabilities and motivations for social interaction, the human "interaction engine". These include special communicative abilities, such as multimodal signal use (Levinson & Holler, 2014) and alternations of speaker-recipient turns in conversation (Levinson, 2016), special cognitive abilities, such as shared intentionality, and other ethological outputs, such as leave-taking rituals (Levinson, 2006a). Taken together, the elements of this engine enable human "cognition-for-interaction" (Levinson, 2006a) in a way that is independent of language, even though language has evolved as one of the primary means by which humans coordinate joint action.…”

“…This approach is designed to encompass results from unimodal research (e.g., gestures only: Fröhlich et al, 2016b) and research concentrating on specific moments of the bout (e.g., opening : Fröhlich et al, 2016b; closing: Luef & Liebal, 2013;main body: Palagi, 2008). The framework takes a multimodal perspective (Levinson & Holler, 2014), including all communicative means, such as gaze, body orientation, behaviors, body postures, facial expressions, vocalizations and gestures and analyzes play bouts in their entirety. This framework can produce a more holistic picture of play coordination and a more complete understanding of how shared intentionality is achieved.…”

Processing language in face-to-face conversation: Questions with gestures get faster responses

“…The primary site of human language use is face-to-face conversation, suggesting that human language should be conceptualized as a fundamentally multimodal phenomenon (e.g., Bavelas & Chovil, 2000;Clark, 1996;Goldin-Meadow, 2003;Kendon, 2004Kendon, , 2014Levinson & Holler, 2014;Mondada, 2016;McNeill, 1992). Bodily signals, in particular, manual gestures, add a significant amount of meaning to what is being said.…”