The sensory representation of causally controlled objects

Abstract

Intentional control over external objects is informed by our sensory experience of them, in a continuous dialogue between action and perception.How such control is represented at the sensory level, however, is not understood. Here we devised a brain machine interface (BMI) task that enabled mice to guide a visual cursor to a target location for reward, using activity in brain areas recorded with widefield calcium imaging. Parietal and higher visual cortical regions were more engaged when expert animals controlled the cursor, but not in naïve mice learning the task. Intentional control enhanced responses: single-cell recordings from parietal cortex indicated that the same visual cursor elicited larger responses when mice controlled it than when they passively viewed it. Moreover, neural responses were greater when the cursor was moving towards the target than away from it. Thus, the sensory representation of a causally-controlled object is sensitive to a subject's intention, as well as the object's instantaneous trajectory relative to the subject's goal: potentially strengthening the sensory feedback signal to adjudicating areas for exerting more fluent control.

Brain–Machine Interfaces: Closed-Loop Control in an Adaptive System

“…In the context of BMIs, it has been used to decode goals and continuous trajectories for reaching and grasping (e.g., 32,33). Animals can learn novel BMI readouts by volitionally controlling PPC activity (34,35).…”

“…Successful BMIs based on calcium imaging have been demonstrated in rodents (35,59,60) and nonhuman primates (61). New innovations in implanted microendoscopes now enable optical imaging in freely behaving nonhuman primates (62).…”

“…Indeed, subjects can control motor cortex neuron activity to use a novel and arbitrary BMI readout (104). Learning volitional control of arbitrary neurons is possible not only in the motor cortex but also in other brain regions, such as the PPC (35) or visual cortex (105). Subjects can also learn to modulate populations of cells together to achieve a more robust interface (106).…”

“…Animal studies have revealed that learning consists of multiple phases governed by different mechanisms (107). Initial practice with a novel BMI entails exploratory and apparently random neural variability (e.g., 35,108), while long-term learning sees the emergence of new internal models that incorporate the device (109,110). BMIs can take advantage of this longer-term learning, and indeed subjects report a transition from deliberate to automatic control of a BMI with practice (111).…”