Deictic codes for the embodiment of cognition

Abstract

To describe phenomena that occur at different time scales, computational models of the brain must incorporate different levels of abstraction. At time scales of approximately 1/3 of a second, orienting movements of the body play a crucial role in cognition and form a useful computational level – more abstract than that used to capture natural phenomena but less abstract than what is traditionally used to study high-level cognitive processes such as reasoning. At this “embodiment level,” the constraints of the physical system determine the nature of cognitive operations. The key synergy is that at time scales of about 1/3 of a second, the natural sequentiality of body movements can be matched to the natural computational economies of sequential decision systems through a system of implicit reference called deictic in which pointing movements are used to bind objects in the world to cognitive programs. This target article focuses on how deictic bindings make it possible to perform natural tasks. Deictic computation provides a mechanism for representing the essential features that link external sensory data with internal cognitive programs and motor actions. One of the central features of cognition, working memory, can be related to moment-by-moment dispositions of body features such as eye movements and hand movements.

Pictures and Spoken Descriptions Elicit Similar Eye Movements During Mental Imagery, Both in Light and in Complete Darkness

“…The eyes could then "leave behind" a deictic pointer or spatial index in the external memory (Altmann, 2004;Ballard et al, 1997;Spivey et al, 2004). In this view, the "scanning" of an image is accomplished by binding the imagined objects onto the actual visual features in the world.…”

“…Spivey and Geng (2001) and Spivey et al (2000) had a different approach than Brandt and Stark (1997) and Laeng and Teodorescu (2002). They interpreted their results in an "embodied" view of the mind (e.g., Ballard, Hayhoe, Pook, & Rao, 1997;Spivey, Richardson, & Fitneva, 2004) whereby motor processes, such as eye movements, are naturally and tightly coupled with the perceptual and cognitive processes that subserve mental representations.…”

Cortical basis of communication: Local computation, coordination, attention

“…Ballard and colleagues [5] have illustrated such a dual view in a task of identification or location of stimuli. From high-dimensional feature vectors extracted from the image of objects, they first explain how a neuronal distribution of activity can be memorized in internal representations.…”

“…Ballard and his colleagues [5] have done a decisive step to operationalize this concept. From the idea that the orientation of the body is important for cognition, they propose that eye movements, performed at an average rate of one third of a second, are the right level of description of the computational interface between neuronal and behavioral processes: the embodiment level.…”

“…Concerning fixation, Ballard and colleagues [5] have deeply studied an experimental procedure, in which a subject has at his disposal a set of colored blocks to copy a model onto a workspace. Behavioral studies reveal the strategy generally chosen by subjects, faithful to the minimal internal memory principle and described by a set of eye movements.…”