Efficient human motion prediction using temporal convolutional generative adversarial network

Cui, Qiongjie; Sun, Huaijiang; Kong, Yong; Zhang, Xiaoqian; Li, Yanmeng

doi:10.1016/j.ins.2020.08.123

Cited by 71 publications

(34 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Completing this task not only is very time-consuming but also requires the animator to be experienced, which greatly limits the development of virtual character dance animation. erefore, a successful dance synthesis algorithm can be useful in areas such as music-assisted dance teaching [5][6][7], audio-visual game character movement generation [8,9], human behavior research [10][11][12][13], and virtual reality.…”

Section: Introductionmentioning

confidence: 99%

[Retracted] Research on the Application of Intelligent Choreography for Musical Theater Based on Mixture Density Network Algorithm

Cang

Huang

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Musical choreography is usually completed by professional choreographers, which is very professional and time-consuming. In order to realize the intelligent choreography of musical, based on the mixed density network (MDN), this paper generates the dance matching with the target music through three steps: motion generation, motion screening, and feature matching. The choreography results in this paper have a high degree of matching with music, which makes it possible for the development of motion capture technology and artificial intelligence and computer automatic choreography based on music. In the process of motion generation, the average value of Gaussian model output by MDN is used as the bone position and the consistency of motion is measured according to the change rate of joint velocity in adjacent frames in the process of motion selection. Compared with the existing studies, the dance generated in this paper has improved in motion coherence and realism. In this paper, a multilevel music and action feature matching algorithm combining global feature matching and local feature matching is proposed. The algorithm improves the unity and coherence of music and action. The algorithm proposed in this paper improves the consistency and novelty of movement, the compatibility with music, and the controllability of dance characteristics. Therefore, the algorithm in this paper technically changes the way of artistic creation and provides the possibility for the development of motion capture technology and artificial intelligence.

show abstract

Section: Introductionmentioning

confidence: 99%

[Retracted] Research on the Application of Intelligent Choreography for Musical Theater Based on Mixture Density Network Algorithm

Cang

Huang

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…To address this limitation, several methods have attempted to better model the distribution of valid motions. In this context, [20] integrates adversarial training to enforce frame-wise geometric plausibility and sequence-wise coherence; [50] uses a GAN with several discriminators that operates on input sequences with masked joints and learns to inpaint the missing information; [14] exploits a similar GAN but uses spectral normalization to perform temporal attention; [61] formulates motion prediction as a generative adversarial imitation learning task to focus on shorter sequences by breaking long ones into small chunks. In contrast to the previous methods that generate a single future prediction, several GAN-and VAE-based methods aim to produce multiple diverse future motion sequences [4,5,8,29,42,67,68].…”

Section: Single-actor Motion Predictionmentioning

confidence: 99%

Dyadic Human Motion Prediction

Katircioglu¹,

Georgantas²,

Salzmann³

et al. 2021

Preprint

View full text Add to dashboard Cite

Prior work on human motion forecasting has mostly focused on predicting the future motion of single subjects in isolation from their past pose sequence. In the presence of closely interacting people, however, this strategy fails to account for the dependencies between the different subject's motions. In this paper, we therefore introduce a motion prediction framework that explicitly reasons about the interactions of two observed subjects. Specifically, we achieve this by introducing a pairwise attention mechanism that models the mutual dependencies in the motion history of the two subjects. This allows us to preserve the long-term motion dynamics in a more realistic way and more robustly predict unusual and fast-paced movements, such as the ones occurring in a dance scenario. To evaluate this, and because no existing motion prediction datasets depict two closelyinteracting subjects, we introduce the LindyHop600K dance dataset. Our results evidence that our approach outperforms the state-of-the-art single person motion prediction techniques.

show abstract

“…What's more, these networks neglect the inner-frame kinematic dependencies between body joints. Generative adversarial networks [53,10,21,12,6,44,23] are deemed to generate realistic data whose pattern is similar to the training data. Nevertheless, they are vulnerable and require skillful training.…”

Section: Related Workmentioning

confidence: 99%

“…Lots of prior efforts with Convolutional Neural Networks (CNNs) [49,28], Recurrent Neural Networks (RNNs) [9,34,41,42,37,11,5,2], and Generative Adversarial Networks (GANs) [53,10,21,12,6,44,23], have been made for tackling the challenging task. However, they neglect the inner-frame kinematic dependencies between body joints.…”

Section: Introductionmentioning

confidence: 99%

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Dang¹,

Nie²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human motion prediction is a challenging task due to the stochasticity and aperiodicity of future poses. Recently, graph convolutional network has been proven to be very effective to learn dynamic relations among pose joints, which is helpful for pose prediction. On the other hand, one can abstract a human pose recursively to obtain a set of poses at multiple scales. With the increase of the abstraction level, the motion of the pose becomes more stable, which benefits pose prediction too. In this paper, we propose a novel Multi-Scale Residual Graph Convolution Network (MSR-GCN) for human pose prediction task in the manner of end-to-end. The GCNs are used to extract features from fine to coarse scale and then from coarse to fine scale. The extracted features at each scale are then combined and decoded to obtain the residuals between the input and target poses. Intermediate supervisions are imposed on all the predicted poses, which enforces the network to learn more representative features. Our proposed approach is evaluated on two standard benchmark datasets, i.e., the Human3.6M dataset and the CMU Mocap dataset. Experimental results demonstrate that our method outperforms the state-of-the-art approaches. Code and pre-trained models are available at https://github.com/Droliven/MSRGCN.

show abstract

Efficient human motion prediction using temporal convolutional generative adversarial network

Cited by 71 publications

References 12 publications

[Retracted] Research on the Application of Intelligent Choreography for Musical Theater Based on Mixture Density Network Algorithm

[Retracted] Research on the Application of Intelligent Choreography for Musical Theater Based on Mixture Density Network Algorithm

Dyadic Human Motion Prediction

MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

Contact Info

Product

Resources

About