Human Motion prediction based on attention mechanism

Sang, Haifeng; Chen, Zi‐Zhen; He, Deqiang

doi:10.1007/s11042-019-08269-7

Cited by 34 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the serialized nature of human motion data, most previous works adopt RNN as backbone [5,6,[13][14][15][16][17][18]22,31,[34][35][36]42]. For example, ERD [13] improves the recurrent layer of LSTM [19] by placing an encoder before it and a decoder after it.…”

Section: Related Workmentioning

confidence: 99%

Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction

Ma¹,

Nie²,

Long³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents a high-quality human motion prediction method that accurately predicts future human poses given observed ones. Our method is based on the observation that a good "initial guess" of the future poses is very helpful in improving the forecasting accuracy. This motivates us to propose a novel two-stage prediction framework, including an init-prediction network that just computes the good guess and then a formal-prediction network that predicts the target future poses based on the guess. More importantly, we extend this idea further and design a multi-stage prediction framework where each stage predicts initial guess for the next stage, which brings more performance gain. To fulfill the prediction task at each stage, we propose a network comprising Spatial Dense Graph Convolutional Networks (S-DGCN) and Temporal Dense Graph Convolutional Networks (T-DGCN). Alternatively executing the two networks helps extract spatiotemporal features over the global receptive field of the whole pose sequence. All the above design choices cooperating together make our method outperform previous approaches by large margins: 6%-7% on Human3.6M, 5%-10% on CMU-MoCap, and 13%-16% on 3DPW. Code is available at https://github.com/705062791/PGBIG.

show abstract

Section: Related Workmentioning

confidence: 99%

Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction

Ma¹,

Nie²,

Long³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, pose data, in essence, is very different from images, lacking repeated elements that give a high response to the same filter, thus reducing the effectiveness of the convolutions. Although RNN-based methods like [9,34,41,42,37,11,5,2] have advantages in dealing with time-related tasks, the discontinuity and error accumulation problems often happen because of the frame-byframe prediction manner. Also, the training of RNN models is easy to collapse with gradient explosion or disappearing.…”

Section: Related Workmentioning

confidence: 99%

“…Lots of prior efforts with Convolutional Neural Networks (CNNs) [49,28], Recurrent Neural Networks (RNNs) [9,34,41,42,37,11,5,2], and Generative Adversarial Networks (GANs) [53,10,21,12,6,44,23], have been made for tackling the challenging task. However, they neglect the inner-frame kinematic dependencies between body joints.…”

Section: Introductionmentioning

confidence: 99%

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Dang¹,

Nie²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human motion prediction is a challenging task due to the stochasticity and aperiodicity of future poses. Recently, graph convolutional network has been proven to be very effective to learn dynamic relations among pose joints, which is helpful for pose prediction. On the other hand, one can abstract a human pose recursively to obtain a set of poses at multiple scales. With the increase of the abstraction level, the motion of the pose becomes more stable, which benefits pose prediction too. In this paper, we propose a novel Multi-Scale Residual Graph Convolution Network (MSR-GCN) for human pose prediction task in the manner of end-to-end. The GCNs are used to extract features from fine to coarse scale and then from coarse to fine scale. The extracted features at each scale are then combined and decoded to obtain the residuals between the input and target poses. Intermediate supervisions are imposed on all the predicted poses, which enforces the network to learn more representative features. Our proposed approach is evaluated on two standard benchmark datasets, i.e., the Human3.6M dataset and the CMU Mocap dataset. Experimental results demonstrate that our method outperforms the state-of-the-art approaches. Code and pre-trained models are available at https://github.com/Droliven/MSRGCN.

show abstract

“…Sang et. al [22] also proposed 2 models: At-seq2seq and seq2seq. In At-seq2seq model with attention mechanism They have used GRU cell in encoder and decoder.…”

Section: A Rnn Based Models Forecasts Human Motion Predictionmentioning

confidence: 99%

Development of Human Motion Prediction Strategy using Inception Residual Block

Gupta,

Yadav,

Nandi

2021

Preprint

View full text Add to dashboard Cite

Human Motion Prediction is a crucial task in computer vision and robotics. It has versatile application potentials such as in the area of human-robot interactions, human action tracking for airport security systems, autonomous car navigation, computer gaming to name a few. However, predicting human motion based on past actions is an extremely challenging task due to the difficulties in detecting spatial and temporal features correctly. To detect temporal features in human poses, we propose an Inception Residual Block(IRB), due to its inherent capability of processing multiple kernels to capture salient features. Here, we propose to use multiple 1-D Convolution Neural Network (CNN) with different kernel sizes and input sequence lengths and concatenate them to get proper embedding. As kernels strides over different receptive fields, they detect smaller and bigger salient features at multiple temporal scales. Our main contribution is to propose a residual connection between input and the output of the inception block to have a continuity between the previously observed pose and the next predicted pose. With this proposed architecture, it learns prior knowledge much better about human poses and we achieve much higher prediction accuracy as detailed in the paper. Subsequently, we further propose to feed the output of the inception residual block as an input to the Graph Convolution Neural Network (GCN) due to its better spatial feature learning capability. We perform a parametric analysis for better designing of our model and subsequently, we evaluate our approach on the Human 3.6M dataset and compare our short-term as well as long-term predictions with the state of the art papers, where our model outperforms most of the pose results, the detailed reasons of which have been elaborated in the paper.

show abstract

Human Motion prediction based on attention mechanism

Cited by 34 publications

References 23 publications

Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction

Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Development of Human Motion Prediction Strategy using Inception Residual Block

Contact Info

Product

Resources

About