A novel pan‐tilt user‐interface (UI) robot system uses a scanning MEMS projection head in combination with a depth sensor. The UI robot system can detect and track a hand, then project a virtual‐remote‐controller (VRC) image onto the top of a palm or table.

Abstract-We used visuomotor tracking as our motor task and studied how subjects learn to adjust for inversion of the relation between joystick movement and target movement. This task requires learning a novel sensorimotor transformation. We have measured tracking performance and pupil dilation simultaneously. We have used pupil dilation as a measure of cognitive load, since the diameter of the human pupil increases with task difficulty across a wide range of cognitive tasks. Subjects observed a target moving at constant velocity along a clockwise circular trajectory on a computer screen. Subjects held a joystick in their hand, and moved it so that a cursor tracked the target as closely as possible. 60 normal subjects participated in the experiment. During 6 blocks of learning, inversion-evoked tracking error and inversionevoked pupil dilation both decreased significantly. This finding suggests increasing automatization of the to-be-learned sensorimotor transformation. Pupil measures were not correlated with tracking error on individual trials, suggesting that the inversion-evoked cognitive load reflects changes in motor task, and is not merely a response to high errors. Our results thus suggest a relatively direct physiological measure of the processes of motor-skill automatization.