Socially and Contextually Aware Human Motion and Pose Forecasting

Adeli, Vida; Adeli, Ehsan; Reid, Ian; Niebles, Juan Carlos; Rezatofighi, Hamid

doi:10.1109/lra.2020.3010742

Cited by 63 publications

(41 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, none of those methods solved the pose forecasting problem when both the trajectory and local pose change. There are still some works in the literature that predicted pose when they move globally [2,1,9,34,46]. Some of them proposed the goal-directed human motion forecasting by incorporating the context of the scene in the prediction [9] or synthesizing human motion between two points given the inputs of start and end positions [46].…”

Section: Pose Forecastingmentioning

confidence: 99%

“…Some of them proposed the goal-directed human motion forecasting by incorporating the context of the scene in the prediction [9] or synthesizing human motion between two points given the inputs of start and end positions [46]. Closer to ours, SC-MPF [1] and TRiPOD [2] predicted the trajectory and local pose dynamics as a single task by considering various human-human, human-objects, and human-scene interactions. In another approach, the human motion was split into the global and local dynamics [34] in a deterministic way.…”

Section: Pose Forecastingmentioning

confidence: 99%

“…The trajectory {R t } t=1:T obs represents the global movements of the human with respect to the camera and the local pose sequence {r t } t=1:T obs indicates the fine-grained movements of keypoints with respect to the trajectory. Similar to previous works [36,1], instead of locations, we use velocities which are more effective in the learning process. The velocity of the pose at time t is simply { Ṗ i t } = P t − P t−1 and its i-th element (i ∈ 1, .…”

Section: Formulationmentioning

confidence: 99%

“…• SC-MPF [1]: It is another recent method for pose forecasting. The human-to-human interaction is included in this model using social pooling.…”

Section: Baselinesmentioning

confidence: 99%

See 3 more Smart Citations

Learning Decoupled Representations for Human Pose Forecasting

Parsaeifard

Saadatnejad

Liu

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

Human pose forecasting involves complex spatiotemporal interactions between body parts (e.g., arms, legs, spine). State-of-the-art approaches use Long Short-Term Memories (LSTMs) or Variational AutoEncoders (VAEs) to solve the problem. Yet, they do not effectively predict human motions when both global trajectory and local pose movements exist. We propose to learn decoupled representations for the global and local pose forecasting tasks. We also show that it is better to stop the prediction when the uncertainty in human motion increases. Our forecasting model outperforms all existing methods on the pose forecasting benchmark to date by over 20%. The code is available online † .

show abstract

Section: Pose Forecastingmentioning

confidence: 99%

Section: Pose Forecastingmentioning

confidence: 99%

Section: Formulationmentioning

confidence: 99%

“…• SC-MPF [1]: It is another recent method for pose forecasting. The human-to-human interaction is included in this model using social pooling.…”

Section: Baselinesmentioning

confidence: 99%

See 2 more Smart Citations

Learning Decoupled Representations for Human Pose Forecasting

Parsaeifard

Saadatnejad

Liu

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

show abstract

“…However, this fails to account for interactions that condition future behavior. Only in [1] is there an attempt to capture them via the use of social cues obtained by pooling the learned features for each individual. While effective in the presence of weak social interactions, this approach is ill-suited to modeling the stronger dependencies that arise from two closely-interacting individuals whose movements are highly correlated.…”

Section: Introductionmentioning

confidence: 99%

Dyadic Human Motion Prediction

Katircioglu¹,

Georgantas²,

Salzmann³

et al. 2021

Preprint

View full text Add to dashboard Cite

Prior work on human motion forecasting has mostly focused on predicting the future motion of single subjects in isolation from their past pose sequence. In the presence of closely interacting people, however, this strategy fails to account for the dependencies between the different subject's motions. In this paper, we therefore introduce a motion prediction framework that explicitly reasons about the interactions of two observed subjects. Specifically, we achieve this by introducing a pairwise attention mechanism that models the mutual dependencies in the motion history of the two subjects. This allows us to preserve the long-term motion dynamics in a more realistic way and more robustly predict unusual and fast-paced movements, such as the ones occurring in a dance scenario. To evaluate this, and because no existing motion prediction datasets depict two closelyinteracting subjects, we introduce the LindyHop600K dance dataset. Our results evidence that our approach outperforms the state-of-the-art single person motion prediction techniques.

show abstract

Semantic Synthesis of Pedestrian Locomotion

Priisalu

Păduraru

Pirinen

et al. 2021

Computer Vision – ACCV 2020

View full text Add to dashboard Cite

It is difficult to perform 3D reconstruction from on-vehicle gathered video due to the large forward motion of the vehicle. Even object detection and human sensing models perform significantly worse on onboard videos when compared to standard benchmarks because objects often appear far away from the camera compared to the standard object detection benchmarks, image quality is often decreased by motion blur and occlusions occur often. This has led to the popularisation of traffic data-specific benchmarks. Recently Light Detection And Ranging (LiDAR) sensors have become popular to directly estimate depths without the need to perform 3D reconstructions. However, LiDAR-based methods still lack in articulated human detection at a distance when compared to image-based methods. We hypothesize that benchmarks targeted at articulated human sensing from LiDAR data could bring about increased research in human sensing and prediction in traffic and could lead to improved traffic safety for pedestrians.

show abstract

Socially and Contextually Aware Human Motion and Pose Forecasting

Cited by 63 publications

References 45 publications

Learning Decoupled Representations for Human Pose Forecasting

Learning Decoupled Representations for Human Pose Forecasting

Dyadic Human Motion Prediction

Semantic Synthesis of Pedestrian Locomotion

Contact Info

Product

Resources

About