Motion Prediction via Joint Dependency Modeling in Phase Space

Abstract

Motion prediction is a classic problem in computer vision, which aims at forecasting future motion given the observed pose sequence. Various deep learning models have been proposed, achieving stateof-the-art performance on motion prediction. However, existing methods typically focus on modeling temporal dynamics in the pose space. Unfortunately, the complicated and high dimensionality nature of human motion brings inherent challenges for dynamic context capturing. Therefore, we move away from the conventional pose based representation and present a novel approach employing a phase space trajectory representation of individual joints. Moreover, current methods tend to only consider the dependencies between physically connected joints. In this paper, we introduce a novel convolutional neural model to effectively leverage explicit prior knowledge of motion anatomy, and simultaneously capture both spatial and temporal information of joint trajectory dynamics. We then propose a global optimization module that learns the implicit relationships between individual joint features.Empirically, our method is evaluated on large-scale 3D human motion benchmark datasets (i.e., Human3.6M, CMU MoCap). These results demonstrate that our method sets the new state-of-the-art on the benchmark datasets. Our code is released at https://github. com/Pose-Group/TEID. CCS CONCEPTS• Computing methodologies → Computer vision; Activity recognition and understanding.

3D Human Motion Prediction: A Survey

“…Mathematically, these parameters are mapped to different mathematical spaces and abstracted into different distributions whose features are easy to be extracted by the network. As is shown in Tab 1, representative papers in this category are: SRNN [36], AGED [28], QuaterNet [79], LTD [67], HMR [60], MGCN [110], ARNet [12], LDR [17], LPJP [11], HRI [66], JDM [91], SGRU [62]. MT-GCN [15], MPTC [57], MST-GNN [47], MMA [68].…”

“…Mathematical representation SRNN [36] 2016 AGED [28] 2018 QuaterNet [79] 2018 LTD [67] 2019 HMR [60] 2019 MGCN [110] 2020 HRI [66] 2020 ARNet [12] 2020 LDR [17] 2021 LPJP [11] 2021 TrajectoryCNN [53] 2021 JDM [91] 2021 MMA [68] 2021 MPTC [57] 2021 SGAN [62] 2021 MT-GCN [15] 2021 MPTC [57] 2021 MST-GNN [47] 2021 skeleton as a graph, which makes GCN widely utilized in human pose representation. Additionally, widespread recognition of the Temporal Convolution Networks (TCNs), made the CN-based methods more widely accepted for solving the sequential problem.…”

“…Additionally, widespread recognition of the Temporal Convolution Networks (TCNs), made the CN-based methods more widely accepted for solving the sequential problem. These methods include: TE [9], PAML [29], CHA [49], LDR [17], MGCN [110], LTD [67], LDR [17], MoPredNet [107], TrajectoryCNN [53], NAT [44], C-seq2seq [45], Q-DCRN [73], MT-GCN [15], LMC [110], MST-GNN [47], DA-GNN [48], JDM [91].…”

“…CN-based methods TE [9] 2017 C-seq2seq [45] 2018 PAML [29] 2018 CHA [49] 2019 LTD [67] 2019 LDR [17] 2020 MGCN [110] 2020 MoPredNet [107] 2020 TrajectoryCNN [53] 2021 NAT [44] 2021 Q-DCRN [73] 2021 MT-GCN [15] 2021 LMC [110], 2021 MST-GNN [47] 2021 DA-GNN [48] 2021 JDM [91] 2021…”

“…The adjacency matrix that represented the weights of the connected joint parts was set as the model parameters. A similar method was also used in [15,91]. Analogously, Mao et al [67] also created a new method to learn an automaticconnected graph.…”