Real-Time Multi-Person Pose Tracking using Data Assimilation

Buizza, Caterina; Fischer, Tobias; Demiris, Yiannis

doi:10.1109/wacv45572.2020.9093442

Cited by 7 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alphapose [11] uses the Pose-Guided Attention (PGA) mechanism to enhance human identity features and integrates human proposal information based on bounding boxes and poses to achieve identity matching. Buizza et al [12] use data assimilation to predict the results of the next frame and realize pose tracking. Algabri et al [44] combine multiple features into a single joint feature and utilize an online enhancement method to continuously update features in each frame for the identification of target individuals.…”

Section: Association Of Identitiesmentioning

confidence: 99%

See 1 more Smart Citation

Bidirectional Temporal Pose Matching for Tracking

Fang,

Shi,

Yang

2024

Electronics

View full text Add to dashboard Cite

Multi-person pose tracking is a challenging task. It requires identifying the human poses in each frame and matching them across time. This task still faces two main challenges. Firstly, sudden camera zooming and drastic pose changes between adjacent frames may result in mismatched poses between them. Secondly, the time relationships modeled by most existing methods provide insufficient information in scenarios with long-term occlusion. In this paper, to address the first challenge, we propagate the bounding boxes of the current frame to the previous frame for pose estimation, and match the estimated results with the previous ones, which we call the Backward Temporal Pose-Matching (BTPM) module. To solve the second challenge, we design an Association Across Multiple Frames (AAMF) module that utilizes long-term temporal relationships to supplement tracking information lost in the previous frames as a Re-identification (Re-id) technique. Specifically, we select keyframes with a fixed step size in the videos and label other frames as general frames. In the keyframes, we use the BTPM module and the AAMF module to perform tracking. In the general frames, we propagate poses in the previous frame to the current frame for pose estimation and association, which we call the Forward Temporal Pose-Matching (FTPM) module. If the pose association fails, the current frame will be set as a keyframe, and tracking will be re-performed. In the PoseTrack 2018 benchmark tests, our method shows significant improvements over the baseline methods, with improvements of 2.1 and 1.1 in mean Average Precision (mAP) and Multi-Object Tracking Accuracy (MOTA), respectively.

show abstract

Section: Association Of Identitiesmentioning

confidence: 99%

“…Some previous multi-person pose-tracking methods benefit from the accuracy advantage brought by MPPE and obtain better tracking results. However, some other multi-person pose-tracking methods [11][12][13][14] do not rely on the selection of MPPE methods but try to improve the accuracy through tracking means.…”

Section: Introductionmentioning

confidence: 99%

Bidirectional Temporal Pose Matching for Tracking

Fang,

Shi,

Yang

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…In [31], the authors suggest a methodology that combines NN and DA for model error correction. Instead, in [32], the authors use DA, in particular Kalman tracking, to speed up any learning-based motion tracking method to real-time and to correct some common inconsistencies in motion tracking methods that are based on the camera. Finally, in [33], the authors introduce a new neural network for speed and replace the whole DA process.…”

Section: Related Work and Contribution Of The Present Workmentioning

confidence: 99%

Deep Data Assimilation: Integrating Deep Learning with Data Assimilation

et al. 2021

View full text Add to dashboard Cite

In this paper, we propose Deep Data Assimilation (DDA), an integration of Data Assimilation (DA) with Machine Learning (ML). DA is the Bayesian approximation of the true state of some physical system at a given time by combining time-distributed observations with a dynamic model in an optimal way. We use a ML model in order to learn the assimilation process. In particular, a recurrent neural network, trained with the state of the dynamical system and the results of the DA process, is applied for this purpose. At each iteration, we learn a function that accumulates the misfit between the results of the forecasting model and the results of the DA. Subsequently, we compose this function with the dynamic model. This resulting composition is a dynamic model that includes the features of the DA process and that can be used for future prediction without the necessity of the DA. In fact, we prove that the DDA approach implies a reduction of the model error, which decreases at each iteration; this is achieved thanks to the use of DA in the training process. DDA is very useful in that cases when observations are not available for some time steps and DA cannot be applied to reduce the model error. The effectiveness of this method is validated by examples and a sensitivity study. In this paper, the DDA technology is applied to two different applications: the Double integral mass dot system and the Lorenz system. However, the algorithm and numerical methods that are proposed in this work can be applied to other physics problems that involve other equations and/or state variables.

show abstract