3D human motion prediction: A survey

Lyu, Kedi; Chen, Haipeng; Liu, Zhenguang; Zhang, Beiqi; Wang, Ruili

doi:10.1016/j.neucom.2022.02.045

Cited by 38 publications

(13 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…distP enalize j = 0 if distJoints j < distJoints j otherwise (10) To make the training more stable we use the Similarity loss only during the first M steps of the training.…”

Section: B Model Trainingmentioning

confidence: 99%

See 1 more Smart Citation

DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network

Nikdel¹,

Mahdavian²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

“…distP enalize j = 0 if distJoints j < distJoints j otherwise (10) To make the training more stable we use the Similarity loss only during the first M steps of the training.…”

Section: B Model Trainingmentioning

confidence: 99%

“…For solving both problems, seq2seq models have been utilized successfully with room for improvement. Predicting a human future motion sequence can be defined as a probabilistic or deterministic problem [10]. In probabilistic methods, similar to how our brain performs, we predict multiple future motion Fig.…”

Section: Introductionmentioning

confidence: 99%

DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network

Nikdel¹,

Mahdavian²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

“…Accurately predicting future motion is a challenging task for classical physical models [5] due to the uncertainty generated by the complexity of human kinematics and the physical environment. The success of deep learning and the emergence of large-scale motion capture datasets have facilitated data-driven research in this field [6][7][8][9][10][11][12][13][14][15][16][17][18]. The human motion sequence can be naturally represented as a collection of joint trajectories equipped with complex spatial-temporal dependencies.…”

Section: Introductionmentioning

confidence: 99%

M2AST:MLP-Mixer-based Adaptive Spatial-Temporal Graph Learning for Human Motion Prediction

Tang

Liu

et al. 2023

Preprint

View full text Add to dashboard Cite

Human motion prediction is a challenging task in human-centric computer vision that involves forecasting future poses based on historical sequences. Despite recent progress in modeling spatial-temporal relationships of motion sequences using complex structured graphs, few approaches have been able to provide an adaptive and compact representation for varying graph structures of human motion. Inspired by the advantages of MLP-Mixer, a lightweight architecture developed for learning complex interactions in multi-dimensional data, we explore its potential as a backbone for motion prediction. Human motion prediction is a challenging task in human-centric computer vision, involving forecasting future poses based on historical sequences. Despite recent progress in modeling spatial-temporal relationships of motion sequences using complex structured graphs, few approaches have provided an adaptive and lightweight representation for varying graph structures of human motion. Taking inspiration from the advantages of MLP-Mixer, a lightweight architecture designed for learning complex interactions in multi-dimensional data, we explore its potential as a backbone for motion prediction. To this end, we propose a novel MLP-Mixer-based adaptive spatial-temporal pattern learning framework (M\(^2\)AST). Our framework includes an adaptive spatial mixer to model the spatial relationships between joints, an adaptive temporal mixer to learn temporal smoothness, and a local dynamic mixer to capture fine-grained cross-dependencies between joints of adjacent poses. The final method achieves a compact representation of human motion dynamics by adaptively considering spatial-temporal dependencies from coarse to fine. Unlike the trivial spatial-temporal MLP-Mixer, our proposed approach can more effectively capture both local and global spatial-temporal relationships simultaneously. We extensively evaluated our proposed framework on three commonly used benchmarks (Human3.6M, AMASS, 3DPW MoCap), demonstrating comparable or better performance than existing state-of-the-art methods in both short and long-term predictions, despite having significantly fewer parameters. Overall, our proposed framework provides a novel and efficient solution for human motion prediction with adaptive graph learning.

show abstract

“…Pedestrians in difference to vehicles provide strong visual cues of their intent, as well as current and future motion through their articulated pose [47][48][49]. Human motion is predictable up to one second with around one centimeter average per joint error when observing articulated motion [50]. The motion information present in the pedestrian pose is unused in most AV motion planning models , as well as in AV model testing.…”

Section: Introductionmentioning

confidence: 99%

“…But pedestrians plan their motion in the scene depending on the geometry of the semantics surrounding them; for example, pedestrians may cross the road to avoid staying on a pavement that is very shallow and is next to a densely trafficked road [48,49]. Further, pedestrian dynamics depend on the particular pedestrian's physique [50]. A complete pedestrian forecasting model should therefore be semantically aware as well as articulated.…”

Section: Introductionmentioning

confidence: 99%

Semantic Synthesis of Pedestrian Locomotion

Priisalu

Păduraru

Pirinen

et al. 2021

Computer Vision – ACCV 2020

View full text Add to dashboard Cite

It is difficult to perform 3D reconstruction from on-vehicle gathered video due to the large forward motion of the vehicle. Even object detection and human sensing models perform significantly worse on onboard videos when compared to standard benchmarks because objects often appear far away from the camera compared to the standard object detection benchmarks, image quality is often decreased by motion blur and occlusions occur often. This has led to the popularisation of traffic data-specific benchmarks. Recently Light Detection And Ranging (LiDAR) sensors have become popular to directly estimate depths without the need to perform 3D reconstructions. However, LiDAR-based methods still lack in articulated human detection at a distance when compared to image-based methods. We hypothesize that benchmarks targeted at articulated human sensing from LiDAR data could bring about increased research in human sensing and prediction in traffic and could lead to improved traffic safety for pedestrians.

show abstract

3D human motion prediction: A survey

Cited by 38 publications

References 78 publications

DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network

DMMGAN: Diverse Multi Motion Prediction of 3D Human Joints using Attention-Based Generative Adverserial Network

M2AST:MLP-Mixer-based Adaptive Spatial-Temporal Graph Learning for Human Motion Prediction

Semantic Synthesis of Pedestrian Locomotion

Contact Info

Product

Resources

About