The Relationship Between Spatial Transformations and Iconic Gestures

Trafton, J. Gregory; Trickett, Susan Bell; Stitzlein, Cara; Saner, Lelyn D.; Schunn, Christian D.; Kirschenbaum, Susan S.

doi:10.1207/s15427633scc0601_1

Cited by 48 publications

(37 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Thus, according to the GSA framework, tasks that require a high amount of active processing should elicit more gestures than should tasks that require passive processing. This is in line with a finding reported by Trafton et al (2005): Gestures occur more often with speech about mental transformations, which involve simulated movement, than with speech about any other spatial property (e.g., size, shape). Future research could test this claim more directly by asking participants to describe information either as it was originally viewed or as it would look after undergoing a mental transformation (e.g., rotation).…”

Section: Predictions Of the Gsa Frameworksupporting

confidence: 92%

Visible embodiment: Gestures as simulated action

Hostetter

Alibali

2008

Psychonomic Bulletin & Review

653

552

View full text Add to dashboard Cite

Gestures are often considered to be valid evidence of the embodiment of language and cognition. Since people use their bodies (i.e., gestures) to express knowledge, it is argued, the knowledge itself must be deeply tied to the body (Alibali & Nathan, 2007;Gallagher, 2005;Gibbs, 2006;Hostetter & Alibali, 2004;McNeill, 2005;Nuñez, 2005). This claim may hold some truth, but to date, there have been no explicit proposals about how embodied knowledge comes to be reflected in speech-accompanying gestures. The goal of this article is to explore how gestures may arise from embodied thinking; we will approach this goal by integrating research about perception, action, cognition, language, mental imagery, and gesture production.A variety of hand and body movements can be considered to be gestures, and much previous research has sought to describe these different types of gestures (e.g., Efron, 1972;Kendon, 2004;McNeill, 1992). In this article, we limit our discussion to representational gestures-that is, movements that represent the content of speech by pointing to a referent in the physical environment (deictic gestures), depicting a referent with the motion or shape of the hands (iconic gestures), or depicting a concrete referent or indicating a spatial location for an abstract idea (metaphoric gestures). We use the term gestures to refer specifically to representational gestures. Other types of gestures, including beat gestures (movements that emphasize the prosody or structure of speech without conveying semantic information) and interactive gestures (movements used to manage turn taking and other aspects of an interaction between multiple speakers), fall outside the scope of this article.Our aim is to describe a mechanism that may possibly give rise to gestures. We do not seek to address the issue of the functions of gestures (e.g., in communication, in speech production). Although we review some ideas about the functions of gestures, we do not take a strong stance here regarding gesture function. The goal of our framework is to explain what gestures are, rather than what they do.The article proceeds in seven sections. In the first section, we review ideas about links between perception and action and how these links are important in memory and cognition. In the second section, we review evidence that language processing is tied to the body. In the third section, we review evidence that mental imagery is also an embodied process. In the fourth section, we review evidence on the relationship between gesture production and mental imagery, and in the fifth, we propose a theoretical framework that explains how gestures may arise from the activation of perception and action. In the sixth section, we compare our framework with other models of gesture production. Finally, in the seventh section, we articulate predictions that derive from the framework. Perception, Action, and Embodied CognitionMany theorists have proposed that perception is for action (J. J. Gibson, 1979;Sperry, 1952); that is, the ability to perceive evolv...

show abstract

Section: Predictions Of the Gsa Frameworksupporting

confidence: 92%

Visible embodiment: Gestures as simulated action

Hostetter

Alibali

2008

Psychonomic Bulletin & Review

653

552

View full text Add to dashboard Cite

show abstract

“…They formulated algorithms faster and more accurately when they could gesture than when they could not. As our theory postulates, when simulations control gestures, they reduce the load on working memory, because they reduce the demands of the task (see also Trafton et al, 2006). It is no longer necessary to describe movements, because iconic gestures communicate them, and likewise the gestures help them to keep track of the positions of cars.…”

Section: Discussionmentioning

confidence: 95%

Children's creation of algorithms: simulations and gestures

Bucciarelli

Mackiewicz

Khemlani

et al. 2016

Journal of Cognitive Psychology

View full text Add to dashboard Cite

for advice.Experiments showed that ten-year old children are able to create algorithms, i.e., sequences of operations that solve problems, and that their gestures help them to do so. The theory of mental models, which is implemented in a computer program, postulates that the creation of algorithms depends on kinematic mental simulations. Gestures are outward signs of moves and they help the process. We tested ten-year children, because they can make mental simulations, and because they gesture more than adults. They readily rearranged the order of six cars in a train (using a siding), and the difficulty of the task depended on the number of moves in minimal solutions (Experiment 1). They were also able to devise informal algorithms to rearrange the order of cars when they were not allowed to move the cars, and the difficulty of the task depended on the complexity of the algorithms (Experiment 2). When children were unable to gesture as they formulated algorithms, the accuracy of their algorithms declined by13% (Experiment 3). We discuss the implications of these results for accounts of reasoning and theories of gestures.

show abstract

“…The mixed orientations of weightless environments, though, may well add an additional challenge for spatial perspective-taking in robots and for their interactive comprehension of astronauts' spatial language. However, virtually all of the experimental work on spatial language and perspective-taking to-date has focused on five frames of reference: exocentric (world-based, such as "Go north"), egocentric (self-based, "Turn to my left"), addressee-centered (other-based, "Turn to your left"), deictic ("Go here [points]"), and object-centric (object-based, "The fork is to the left of the plate") [20]- [26]. Thus, in our analysis, we used this framework to explore the type and amount of spatial perspective-taking that arose among the astronauts in training.…”

Section: Security Classification Of: 17 Limitation Of Abstractmentioning

confidence: 99%

Enabling Effective Human–Robot Interaction Using Perspective-Taking in Robots

Trafton

Cassimatis

Bugajska

et al. 2005

IEEE Trans. Syst., Man, Cybern. A

242

123

View full text Add to dashboard Cite

Abstract-We propose that an important aspect of human-robot interaction is perspective-taking. We show how perspective-taking occurs in a naturalistic environment (astronauts working on a collaborative project) and present a cognitive architecture for performing perspective-taking called Polyscheme. Finally, we show a fully integrated system that instantiates our theoretical framework within a working robot system. Our system successfully solves a series of perspective-taking problems and uses the same frames of references that astronauts do to facilitate collaborative problem solving with a person.

show abstract

The Relationship Between Spatial Transformations and Iconic Gestures

Cited by 48 publications

References 45 publications

Visible embodiment: Gestures as simulated action

Visible embodiment: Gestures as simulated action

Children's creation of algorithms: simulations and gestures

Enabling Effective Human–Robot Interaction Using Perspective-Taking in Robots

Contact Info

Product

Resources

About