Application of Graph Convolutions in a Lightweight Model for Skeletal Human Motion Forecasting

Hermes, Luca; Hammer, Barbara; Schilling, Malte

doi:10.48550/arxiv.2110.04810

Cited by 1 publication

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For other recently proposed methods proposed in refs. [22,29,30,34,39,40], STTG-Net achieved optimal results on more than half of the actions and achieved approximate optimal results on the others. In the comparison of the average error, in addition to the sub-optimal results at 80 ms, STTG-Net achieved the best results at 160, 320, and 400 ms, respectively.…”

Section: Resultsmentioning

confidence: 97%

“…Consistent with previous studies, the model was trained using 50 frames and predicted the pose for the next 10 frames. Table 1 [3,14,22,27,29,30,34,39,40] shows the joint angle error results of all actions compared with baselines of this model on Human3.6 M. In order to observe the results more intuitively, the best results among all the experimental results are presented in bold, and the sub-optimal results are presented in italics.…”

Section: Resultsmentioning

confidence: 99%

“…In order to change the end-to-end training method of the human motion prediction task, Wang et al [25] transformed it into a reinforcement learning problem by proposing a reinforcement learning formulation and an imitation learning algorithm that extended the generative adversarial imitation learning framework to be able to make accurate predictions of poses. [29], who designed a spatio-temporal convolution with a GCN to extract spatio-temporal features, using an expanded causal convolution to model temporal information, which also contains local joint connectivity, to obtain a lightweight autoregressive model. In contrast, Martínez-González et al [30] proposed a non-autoregressive transformer model to infer pose sequences in parallel, with selfattention and encoder-decoder attention, and added a skeleton-based activity classification to the encoder to improve motion prediction through action recognition.…”

Section: The Other Methodsmentioning

confidence: 99%

“…The proposed approach was compared with commonly used motion prediction baselines and some of the latest methods, including Multi-Gan [39], OoD [40], ST-Conv [29], POTR-GCN [30] and ST-transformer [34] as well as the state-of-the-art methods HRI [27] and DMGNN [22].…”

Section: Baselinesmentioning

confidence: 99%

See 3 more Smart Citations

STTG-net: a Spatio-temporal network for human motion prediction based on transformer and graph convolution network

Chen

Liu

Yang

et al. 2022

Vis. Comput. Ind. Biomed. Art

View full text Add to dashboard Cite

In recent years, human motion prediction has become an active research topic in computer vision. However, owing to the complexity and stochastic nature of human motion, it remains a challenging problem. In previous works, human motion prediction has always been treated as a typical inter-sequence problem, and most works have aimed to capture the temporal dependence between successive frames. However, although these approaches focused on the effects of the temporal dimension, they rarely considered the correlation between different joints in space. Thus, the spatio-temporal coupling of human joints is considered, to propose a novel spatio-temporal network based on a transformer and a gragh convolutional network (GCN) (STTG-Net). The temporal transformer is used to capture the global temporal dependencies, and the spatial GCN module is used to establish local spatial correlations between the joints for each frame. To overcome the problems of error accumulation and discontinuity in the motion prediction, a revision method based on fusion strategy is also proposed, in which the current prediction frame is fused with the previous frame. The experimental results show that the proposed prediction method has less prediction error and the prediction motion is smoother than previous prediction methods. The effectiveness of the proposed method is also demonstrated comparing it with the state-of-the-art method on the Human3.6 M dataset.

show abstract