Stable readout of observed actions from format-dependent activity of monkey’s anterior intraparietal neurons

Abstract

Humans accurately identify observed actions despite large dynamic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others’ actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also contributes to invariant coding of OMAs across different visual formats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor’s body postures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor’s body posture emerged shortly after stimulus presentation, followed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depending on the visual specificities of each format. These findings suggest that the AIP integrates format-dependent information and the visual features of others’ actions, leading to a stable readout of observed manipulative action identity.

A parietal region processing numerosity of observed actions: An FMRI study

“…(2017). In the monkey, this rostral part of the lower bank of STS region projects to AIP (Lanzilotto et al, 2019), and the visual signals related to hand–target relationships probably enter into the definition of the visual identity of OMAs (combination of the observed goal of the action and perceived body movements bringing about this result) at that level (Lanzilotto et al, 2020). AIP neurons are tuned to OMAs (Lanzilotto et al, 2019), and if those tuned to flick and push are equal in number, little difference between the two action conditions is expected.…”

“…Categorical distinctions of observed actions (OAs) have been found in LOTC, in particular the abstract action categories transitivity and sociality (Wurm, Caramazza, & Lingnau, 2017), and more recently, the action components such as body parts, scenes, movements, objects, sociality, and transitivity (Tucciarelli, Wurm, Baccolo, & Lingnau, 2019). On the other hand, recent evidence (Lanzilotto et al, 2019(Lanzilotto et al, , 2020 indicates that PPC regions process the visual identity of OAs, in a similar way to the processing by the ventral pathway of the visual identity of objects (Hung, Kreiman, Poggio, & DiCarlo, 2005), and in particular of faces (Chang & Tsao, 2017). By visual identity of OAs, we refer to the integration of the goal of the action, that is, the change in the outside world it aims to produce, and the body movements of the conspecific that allow this goal to be reached.…”

What happened to mirror neurons?

“…For example, individual neurons responsive to the observation of others' manipulative actions (grasping, dragging, etc.) in anterior intraparietal area show viewpoint-dependent coding, but as a population they provide viewpoint-invariant coding of the observed action (Lanzilotto et al, 2020;see also Livi et al, 2019, for a similar demonstration of population encoding of observed actions in presupplementary motor area).…”

A shared neural substrate for action verbs and observed actions in human posterior parietal cortex

“…Videos were presented in a pseudorandom manner: All conditions were randomly ordered and presented once before repetition. Video stimuli used for L0, L1, and F0 were also used in (44), which tested neural encoding of observed actions in nonhuman primates. Presentation of the videos differed, however, as before video presentation, during baseline, this study used a highly blurred (full width at half maximum = 80 pixels) static frame (average of all video frames).…”