Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions

Abstract

The visual recognition of actions is an important visual function that is critical for motor learning and social communication. Actionselective neurons have been found in different cortical regions, including the superior temporal sulcus, parietal and premotor cortex. Among those are mirror neurons, which link visual and motor representations of body movements. While numerous theoretical models for the mirror neuron system have been proposed, the computational basis of the visual processing of goal-directed actions remains largely unclear. While most existing models focus on the possible role of motor representations in action recognition, we propose a model showing that many critical properties of action-selective visual neurons can be accounted for by well-established visual mechanisms. Our model accomplishes the recognition of hand actions from real video stimuli, exploiting exclusively mechanisms that can be implemented in a biologically plausible way by cortical neurons. We show that the model provides a unifying quantitatively consistent account of a variety of electrophysiological results from action-selective visual neurons. In addition, it makes a number of predictions, some of which could be confirmed in recent electrophysiological experiments.

Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations

“…This requires a modeling of additional computations, potentially realized parietal areas that analyze the shape and position of goal objects and the relationship between object and the effector movements. One model of this type reproduces a variety of electrophysiological results from action-selective neurons in the STS and premotor area F5, at the same time accomplishing recognition from real videos (Fleischer et al, 2013) (see Figure 2B). This model reproduces, for example, the view dependence of action-selective neurons (see ''Some Newly Established Properties of Mirror Neurons in Area F5''), their temporal sequence selectivity, and predicts a relationship between mirror neurons and mechanisms for causality perception (Fleischer et al, 2012).…”

“…It comprises three modules: (a) shape recognition hierarchy that recognizes hand and object shapes; (b) ''affordance module'' that analyzes the matching between grip type and objects shape and their spatial parameters; (c) recognition module that consists of neurons that are selective for goal-directed hand actions. View independence is generated at the highest level of the hierarchy by pooling the output signals from viewspecific modules (modified from Fleischer et al, 2013).…”

Grasping Execution and Grasping Observation Activity of Single Neurons in the Macaque Anterior Intraparietal Area

“…Our data are consistent with a role for AIP in the mirror neuron network (Nelissen et al, 2011;Rizzolatti & Sinigaglia, 2010;Oztop & Arbib, 2002) but do not permit to decide whether GO activity in AIP is in fact related to the mirror neuron system or to action recognition in general. Future studies should test whether GO activity in AIP constitutes an early stage in which the mapping between visual and motor properties occurs to provide input for cortical areas involved in the recognition of biological actions and/or mirror activity (Fleischer, Caggiano, Thier, & Giese, 2013).…”

Visual categorization of social interactions

“…The visual recognition of social interactions is, therefore, critical for humans to navigate through both their physical and social environment. So far, prominent theories of action recognition assume that during action recognition the physical pattern of an action is associated with one particular action interpretation (e.g., Fleischer, Caggiano, Thier, & Giese, 2013;Giese & Poggio, 2003;Rizzolatti, Fogassi, & Gallese, 2001). Here we examine whether humans associate different semantic interpretations with the same action.…”