Context-Aware Human Motion Prediction

Abstract

The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision. Current state-ofthe-art formulate this problem as a sequence-to-sequence task, in which a historical of 3D skeletons feeds a Recurrent Neural Network (RNN) that predicts future movements, typically in the order of 1 to 2 seconds. However, one aspect that has been obviated so far, is the fact that human motion is inherently driven by interactions with objects and/or other humans in the environment. In this paper, we explore this scenario using a novel context-aware motion prediction architecture. We use a semantic-graph model where the nodes parameterize the human and objects in the scene and the edges their mutual interactions. These interactions are iteratively learned through a graph attention layer, fed with the past observations, which now include both object and human body motions. Once this semantic graph is learned, we inject it to a standard RNN to predict future movements of the human/s and object/s. We consider two variants of our architecture, either freezing the contextual interactions in the future of updating them. A thorough evaluation in the "Whole-Body Human Motion Database" [29] shows that in both cases, our context-aware networks clearly outperform baselines in which the context information is not considered.

GanHand: Predicting Human Grasp Affordances in Multi-Object Scenes

“…Our work is more related to those approaches that learn human affordances in 3D indoor environments [24,67,39,16]. These works, besides understanding the functionality of the visible elements in an image, predict valid -but coarse -3D human poses and actions within it.…”

Learning Decoupled Representations for Human Pose Forecasting

“…Existing works on pose forecasting mostly ignore the global motion of the human and only predict the changes in keypoints locations with respect to the center of human with the global motion excluded [18,24,36,11,19,13,35,14,45,5,49]. RNNs, capable of capturing the temporal dependencies in sequential data, have been widely used for the problem of local pose forecasting [18,24,36,11,19,13,35].…”

Attention deep learning based model for predicting the 3D Human Body Pose using the Robot Human Handover Phases

“…This is commonly the most studied approach, used in [9], [12], [1]. The approach from [6] is specially interesting, since the model predictions are conditioned on the objects around the humans, such as tables or doors.…”