Low-Dimensional Representations of Visuomotor Coordination in Natural Behavior

Abstract

Understanding how the eyes, head, and hands coordinate in natural contexts is a critical challenge in visuomotor coordination research, often limited by sedentary tasks in constrained settings. To address this gap, we conducted an experiment where participants proactively performed pick-and-place actions on a life-size shelf in a virtual environment and recorded concurrent gaze and body movements. Subjects exhibited intricate translation and rotation movements of the eyes, head, and hands during the task. We employed time-wise principal component analysis to study the relationship between the eye, head, and hand movements relative to the action onset. We reduced the overall dimensionality into 2D representations, capturing over 50% of the explained variance and up to 65% just in time of the actions. Our analysis revealed a synergistic coupling of the eye-head and eye-hand systems. While generally loosely coupled, they synchronized at the moment of action, with variations in coupling observed in horizontal and vertical planes, indicating distinct mechanisms for coordination in the brain. Crucially, the head and hand were tightly coupled throughout the observation period, suggesting a common neural code driving these effectors. Notably, the low-dimensional representations demonstrated maximum predictive accuracy ~200ms before the action onset, highlighting a just-in-time coordination of the three effectors. This study emphasizes the synergistic nature of visuomotor coordination in natural behaviors, providing insights into the dynamic interplay of eye, head, and hand movements during reach-to-grasp tasks.