A Neural Temporal Model for Human Motion Prediction

Abstract

We propose novel neural temporal models for predicting and synthesizing human motion, achieving state-of-theart in modeling long-term motion trajectories while being competitive with prior work in short-term prediction, with significantly less required computation. Key aspects of our proposed system include: 1) a novel, two-level processing architecture that aids in generating planned trajectories, 2) a simple set of easily computable features that integrate derivative information into the model, and 3) a novel multi-objective loss function that helps the model to slowly progress from the simpler task of next-step prediction to the harder task of multi-step closed-loop prediction. Our results demonstrate that these innovations facilitate improved modeling of long-term motion trajectories. Finally, we propose a novel metric, called Normalized Power Spectrum Similarity (NPSS), to evaluate the long-term predictive ability of motion synthesis models, complementing the popular mean-squared error (MSE) measure of the Euler joint angles over time. We conduct a user study to determine if the proposed NPSS correlates with human evaluation of longterm motion more strongly than MSE and find that it indeed does.

Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations

“…state-of-the-art performance in many important tasks that are sequential in nature. These tasks range from those in statistical machine translation [6], to language modeling and text processing [7], [8], [9], to long and short-term human motion generation [10], to speech recognition [11]. To train these powerful recurrent networks, back-propagation through time (BPTT) has long been the the primary algorithm of choice for computing parameter updates.…”

Human Motion Prediction Via Pattern Completion in Latent Representation Space

“…To include action label information, we concatenate a onehot encoded action type vector with each pose, similar to recent literature [6], [8], [21]. With the action label and human motion learned by our autoencoder, this knowledge can be used to solve the action classification task.…”

“…We follow the same evaluation method for short-term prediction as in [4]- [8], [10], [11], [21]. We cite the results from the most relevant works to compare with our method, which are Res-Seq2Seq [6], the model by Tang et al [11], VGRU-rl [21] and AGED [8] which is the current state of the art. We also compare against the naive zero-velocity baseline proposed by [6] and use their code to generate long-term predictions.…”

“…We use two normalization methods depending on the baseline that we are comparing against: (1) subtract the mean and normalized between -1 and 1, which is used in [11], and (2) subtract the mean and divide by the standard deviation, which is used by the other methods [6], [8], [21]. Table I: Comparison of mean angle error between our method and top performing baselines for short-term motion prediction.…”

A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction

“…To preserve the motion trend in long-term prediction, we propose a bi-directional predictor enhanced by a bidiscriminator to adversarially revise the generated forward and backward motion dynamics. From a single forward predictor [8], [10], [17], prediction errors are rapidly accumulated along the temporal domain since RNN models fail to keep the long-term knowledge in recurrent steps, causing the generated motion drifting to a wrong direction. To this end, we train a backward predictor to encode the velocity in reversed timesteps, such that the model recovers the context from the beginning dynamics that are lost during long sequence transition.…”