Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking

Abstract

Multi-person pose estimation and tracking serve as crucial steps for video understanding. Most state-of-the-art approaches rely on first estimating poses in each frame and only then implementing data association and refinement. Despite the promising results achieved, such a strategy is inevitably prone to missed detections especially in heavilycluttered scenes, since this tracking-by-detection paradigm is, by nature, largely dependent on visual evidences that are absent in the case of occlusion. In this paper, we propose a novel online approach to learning the pose dynamics, which are independent of pose detections in current fame, and hence may serve as a robust estimation even in challenging scenarios including occlusion. Specifically, we derive this prediction of dynamics through a graph neural network (GNN) that explicitly accounts for both spatialtemporal and visual information. It takes as input the historical pose tracklets and directly predicts the corresponding poses in the following frame for each tracklet. The predicted poses will then be aggregated with the detected poses, if any, at the same frame so as to produce the final pose, potentially recovering the occluded joints missed by the estimator. Experiments on PoseTrack 2017 and Pose-Track 2018 datasets demonstrate that the proposed method achieves results superior to the state of the art on both human pose estimation and tracking tasks.

Detection of Key Points in Mice at Different Scales via Convolutional Neural Network

“…CPM [15,16] uses a serialized convolution architecture to express spatial information and texture information. Its network structure is divided into multiple stages.…”

“…The interaural distances at different scales show obvious differen normally distributed at the same scale, which is relatively stable and m ments of the scale factor. Finally, there is an issue regarding the non-vi ears in the images; in this case, the median of the inter-ear distance at th is used as a scale factor, i.e., 15.29 at 60 cm, 12.85 at 70 cm, and 10.81 at 8 6 reveals that the scale factor is concentrated in [10,15] pixels. The minimum unit is 1 pixel, and the corresponding distinguishable threshold is 0.1.…”

“…The size of the threshold in the PCK indicator should change with the change in the scale factor. The histogram shown in Figure6reveals that the scale factor is concentrated in[10,15] pixels. The minimum unit is 1 pixel, and the corresponding distinguishable threshold is 0.1.…”