Necessity is the Mother of Invention: Reconstructing Missing Sensory Information in Multiple, Concurrent Reference Frames for Eye–Hand Coordination

Abstract

When aligning the hand to grasp an object, the CNS combines multiple sensory inputs encoded in multiple reference frames. Previous studies suggest that when a direct comparison of target and hand is possible via a single sensory modality, the CNS avoids performing unnecessary coordinate transformations that add noise. But when target and hand do not share a common sensory modality (e.g., aligning the unseen hand to a visual target), at least one coordinate transformation is required. Similarly, body movements may occur between target acquisition and manual response, requiring that egocentric target information be updated or transformed to external reference frames to compensate. Here, we asked subjects to align the hand to an external target, where the target could be presented visually or kinesthetically and feedback about the hand was visual, kinesthetic, or both. We used a novel technique of imposing conflict between external visual and gravito-kinesthetic reference frames when subjects tilted the head during an instructed memory delay. By comparing experimental results to analytical models based on principles of maximum likelihood, we showed that multiple transformations above the strict minimum may be performed, but only if the task precludes a unimodal comparison of egocentric target and hand information. Thus, for cross-modal tasks, or when head movements are involved, the CNS creates and uses both kinesthetic and visual representations. We conclude that the necessity of producing at least one coordinate transformation activates multiple, concurrent internal representations, the functionality of which depends on the alignment of the head with respect to gravity.

When Kinesthesia Becomes Visual: A Theoretical Justification for Executing Motor Tasks in Visual Space

“…pointing with one hand to the other) and even when no visual feedback about the hand is allowed [1]–[5]. This is in apparent contrast with our own studies on human sensori-motor integration [6], [7] in which we observed that if subjects were asked to align their hidden hand to the orientation of a kinesthetically felt target, they completely ignored the information related to the visual scene, indicating that the brain executes purely kinesthetic tasks (K-K: kinesthetic target and kinesthetic response) without using a visual representation of the movement. This apparent contradiction, however, could be related to a fundamental difference between the motor tasks that the subjects were asked to perform in these different sets of studies.…”

“…in a replication of the K-K condition of our previous study [6]) and we compared their responses in two different conditions (Figure 1). In the first, called INTRA-manual, subjects memorized and reproduced the orientation with the same hand.…”

“…In both INTER and INTRA, we exploited the peculiarity of our reach-to-grab paradigm in virtual reality that allows us to assess the weight given to visual information even when the subject had only kinesthetic sensory inputs to control the movement [6], [7]. Specifically, subjects were asked to tilt the head laterally after the target memorization and during the head movement of half of the trials we introduced an imperceptible sensory conflict between the orientation of the visual scene and the direction of gravity.…”

A key region in the human parietal cortex for processing proprioceptive hand feedback during reaching movements

“…Furthermore, it has been suggested that the 'mode of control' of reaching movements to a visual target depends on whether only proprioceptive information about the hand position can be used, or whether vision is also available (Krakauer et al, 1999;Reichenbach et al, 2009). The latter allows for directly relating visual target and hand position in a common reference frame (Tagliabue and McIntyre, 2011), and different neural circuits may be recruited dependent on the mode of control (Bernier et al, 2009). Recent fMRI studies have assessed the relative contribution of different PPC areas to the processing of visual and proprioceptive information during motor control, mainly by investigating brain activity during the whole movement (Bernier and Grafton, 2010;Blangero et al, 2009;Filimon et al, 2009;Pellijeff et al, 2006), in which different temporal stages of movement control can hardly be disentangled.…”

Head roll influences perceived hand position

“…To do so, it is believed that all sensory signals have to be converted into a common frame of reference (Buneo & Andersen, 2006;Cohen & Andersen, 2002;Engel, Flanders, & Soechting, 2002;Knudsen, du Lac, & Esterly, 1987;Lacquaniti & Caminiti, 1998;Soechting & Flanders, 1989). However, reference frame transformations can induce noise that depends on the size of the transformation required (Blohm & Crawford, 2007;Burns & Blohm, 2010;Soechting & Flanders, 1989;Tagliabue & McIntyre, 2011;van Beers, Baraduc, & Wolpert, 2002).…”