Temporal cortical neurons are known to respond to visual dynamic-action displays. Many human psychophysical and functional imaging studies examining biological motion perception have used treadmill walking, in contrast to previous macaque single-cell studies. We assessed thecodingoflocomotioninrhesusmonkey(Macacamulatta)temporalcortexusingmoviesofstationarywalkers,varyingbothformandmotion (i.e.,differentfacingdirections)orvaryingonlytheframesequence(i.e.,forwardvsbackwardwalking).Themajorityofsuperiortemporalsulcus and inferior temporal neurons were selective for facing direction, whereas a minority distinguished forward from backward walking. Support vector machines using the temporal cortical population responses as input classified facing direction well, but forward and backward walking less so. Classification performance for the latter improved markedly when the within-action response modulation was considered, reflecting differences in momentary body poses within the locomotion sequences. Responses to static pose presentations predicted the responses during the course of the action. Analyses of the responses to walking sequences wherein the start frame was varied across trials showed that some neurons also carried a snapshot sequence signal. Such sequence information was present in neurons that responded to static snapshot presentationsandinneuronsthatrequiredmotion.Ourdatasuggestthatactionsareanalyzedbytemporalcorticalneuronsusingdistinctmechanisms. Most neurons predominantly signal momentary pose. In addition, temporal cortical neurons, including those responding to static pose, are sensitive to pose sequence, which can contribute to the signaling of learned action sequences.

A vast literature exists on human biological motion perception in impoverished displays, e.g., point-light walkers. Less is known about the perception of impoverished biological motion displays in macaques. We trained 3 macaques in the discrimination of facing direction (left versus right) and forward versus backward walking using motion-capture-based locomotion displays (treadmill walking) in which the body features were represented by cylinder-like primitives. The displays did not contain translatory motion. Discriminating forward versus backward locomotion requires motion information while the facing-direction/view task can be solved using motion and/or form. All monkeys required lengthy training to learn the forward-backward task, while the view task was learned more quickly. Once acquired, the discriminations were specific to walking and stimulus format but generalized across actors. Although the view task could be solved using form cues, there was a small impact of motion. Performance in the forward-backward task was highly susceptible to degradations of spatiotemporal stimulus coherence and motion information. These results indicate that rhesus monkeys require extensive training in order to use the intrinsic motion cues related to forward versus backward locomotion and imply that extrapolation of observations concerning human perception of impoverished biological motion displays onto monkey perception needs to be made cautiously.

We present a method to simultaneously estimate 3D body pose and action categories from monocular video sequences. Our approach learns a generative model of the relationship of body pose and image appearance using a sparse kernel regressor. Body poses are modelled on a low-dimensional manifold obtained by Locally Linear Embedding dimensionality reduction. In addition, we learn a prior model of likely body poses and a dynamical model in this pose manifold. Sparse kernel regressors capture the nonlinearities of this mapping efficiently. Within a Recursive Bayesian Sampling framework, the potentially multimodal posterior probability distributions can then be inferred. An activity-switching mechanism based on learned transfer functions allows for inference of the performed activity class, along with the estimation of body pose and 2D image location of the subject. Using a rough foreground segmentation, we compare Binary PCA and distance transforms to encode the appearance. As a postprocessing step, the globally optimal trajectory through the entire sequence is estimated, yielding a single pose estimate per frame that is consistent throughout the sequence. We evaluate the algorithm on challenging sequences with subjects that are alternating between running and walking movements. Our experiments show how the dynamical model helps to track through poorly segmented low-resolution image sequences where tracking otherwise fails, while at the same time reliably classifying the activity type.

No abstract

No abstract