Occluded Joints Recovery in 3D Human Pose Estimation based on Distance Matrix

Guo, Xiang; Dai, Yuchao

doi:10.1109/icpr.2018.8545226

Cited by 13 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, to evaluate the effectiveness of our method in tackling the issues of skeleton loss and misidentification within nursing care task scenarios, we conducted a comprehensive performance comparison against several existing methods, including that of Tsai et al [23], a left-right skeletal symmetry skeleton compensation method; Guo et al [24], a Euclidean distance matrix skeleton compensation method; and Kanazawa et al [25], a Human-Dynamics-based temporal skeleton compensation method. The evaluation Based on the comprehensive results presented in Table 6, notable differences (p < 0.001) were observed in the joints scores and REBA scores between the OpenPose and the ground truth values (E REBA1 ), except for Trunk (p = 0.788) and Neck (p = 0.124).…”

Section: Reba Score Errormentioning

confidence: 99%

“…Additionally, when employing Mask RCNN to confine the boundaries of compensated skeletal points in scenes with multiple individuals, the accuracy of pose skeleton estimation is not ideal enough [23]. Existing methods that compensate for occluded skeletons based on a Euclidean distance matrix [24] or that predict future pose skeletons using Human Dynamics [25] share a common limitation: they fail to address the problem of skeletal misidentification, leading to a uniform compensation approach for both correctly identified and misidentified skeletons. Consequently, the compensated skeletons fail to match the target pose skeleton, exacerbating differences in pose skeleton angles and REBA scores.…”

Section: Main Findings and Contributionsmentioning

confidence: 99%

“…Nonetheless, compensating for skeletal keypoints using the symmetry principle often encounters challenges when dealing with complex movements. To restore occluded keypoints, researchers have explored the utilization of unoccluded skeletal keypoints in a Euclidean distance matrix [24]. This skeleton compensation method has proven successful in mitigating skeletal occlusion issues.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Compensation Method for Missing and Misidentified Skeletons in Nursing Care Action Assessment by Improving Spatial Temporal Graph Convolutional Networks

Han,

Nishida,

Morita

et al. 2024

Bioengineering

View full text Add to dashboard Cite

With the increasing aging population, nursing care providers have been facing a substantial risk of work-related musculoskeletal disorders (WMSDs). Visual-based pose estimation methods, like OpenPose, are commonly used for ergonomic posture risk assessment. However, these methods face difficulty when identifying overlapping and interactive nursing tasks, resulting in missing and misidentified skeletons. To address this, we propose a skeleton compensation method using improved spatial temporal graph convolutional networks (ST-GCN), which integrates kinematic chain and action features to assess skeleton integrity and compensate for it. The results verified the effectiveness of our approach in optimizing skeletal loss and misidentification in nursing care tasks, leading to improved accuracy in calculating both skeleton joint angles and REBA scores. Moreover, comparative analysis against other skeleton compensation methods demonstrated the superior performance of our approach, achieving an 87.34% REBA accuracy score. Collectively, our method might hold promising potential for optimizing the skeleton loss and misidentification in nursing care tasks.

show abstract

Section: Reba Score Errormentioning

confidence: 99%

Section: Main Findings and Contributionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Compensation Method for Missing and Misidentified Skeletons in Nursing Care Action Assessment by Improving Spatial Temporal Graph Convolutional Networks

Han,

Nishida,

Morita

et al. 2024

Bioengineering

View full text Add to dashboard Cite

show abstract

“…To deal with partial occlusions, some techniques have been designed to recover occluded keypoints from unoc-cluded ones according to the spatial or temporal context (Radwan, Dhall, and Goecke 2013;Rogez, Weinzaepfel, and Schmid 2017;de Bem et al 2018;Guo and Dai 2018;Cheng et al 2019) or scene constraints . Some methods further introduced the concept of "human dynamics" (Kanazawa et al 2019;Zhang et al 2019), which predicts future human poses according to single or multiple existing frames in a video without any future frames.…”

Section: Related Workmentioning

confidence: 99%

3D Human Pose Estimation Using Spatio-Temporal Networks with Explicit Occlusion Training

Cheng

Yang

Wang

et al. 2020

AAAI

104

View full text Add to dashboard Cite

Estimating 3D poses from a monocular video is still a challenging task, despite the significant progress that has been made in the recent years. Generally, the performance of existing methods drops when the target person is too small/large, or the motion is too fast/slow relative to the scale and speed of the training data. Moreover, to our knowledge, many of these methods are not designed or trained under severe occlusion explicitly, making their performance on handling occlusion compromised. Addressing these problems, we introduce a spatio-temporal network for robust 3D human pose estimation. As humans in videos may appear in different scales and have various motion speeds, we apply multi-scale spatial features for 2D joints or keypoints prediction in each individual frame, and multi-stride temporal convolutional networks (TCNs) to estimate 3D joints or keypoints. Furthermore, we design a spatio-temporal discriminator based on body structures as well as limb motions to assess whether the predicted pose forms a valid pose and a valid movement. During training, we explicitly mask out some keypoints to simulate various occlusion cases, from minor to severe occlusion, so that our network can learn better and becomes robust to various degrees of occlusion. As there are limited 3D ground truth data, we further utilize 2D video data to inject a semi-supervised learning capability to our network. Experiments on public data sets validate the effectiveness of our method, and our ablation studies show the strengths of our network's individual submodules.

show abstract

“…Noguer et al [26] propose the Euclidean Distance Matrice (EDM) between 2D keypoints to estimate the EDM between 3D human joints, finally leading to the human pose. Guo et al [27] based on [26], recover the occluded joints before estimating 3D EDM. Gao et al [28] noticed the distance relationship between non-adjacent joints when determining the distance matrix.…”

Section: Related Workmentioning

confidence: 99%

Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos

Wang¹,

Honghao²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Real-time 3D human pose estimation is crucial for human-computer interaction. It is cheap and practical to estimate 3D human pose only from monocular video. However, recent bone splicing based 3D human pose estimation method brings about the problem of cumulative error. In this paper, the concept of virtual bones is proposed to solve such a challenge. The virtual bones are imaginary bones between non-adjacent joints. They do not exist in reality, but they bring new loop constraints for the estimation of 3D human joints. The proposed network in this paper predicts real bones and virtual bones, simultaneously. The final length of real bones is constrained and learned by the loop constructed by the predicted real bones and virtual bones. Besides, the motion constraints of joints in consecutive frames are considered. The consistency between the 2D projected position displacement predicted by the network and the captured real 2D displacement by the camera is proposed as a new projection consistency loss for the learning of 3D human pose. The experiments on the Human3.6M dataset demonstrate the good performance of the proposed method. Ablation studies demonstrate the effectiveness of the proposed inter-frame projection consistency constraints and intra-frame loop constraints.

show abstract

Occluded Joints Recovery in 3D Human Pose Estimation based on Distance Matrix

Cited by 13 publications

References 23 publications

Compensation Method for Missing and Misidentified Skeletons in Nursing Care Action Assessment by Improving Spatial Temporal Graph Convolutional Networks

Compensation Method for Missing and Misidentified Skeletons in Nursing Care Action Assessment by Improving Spatial Temporal Graph Convolutional Networks

3D Human Pose Estimation Using Spatio-Temporal Networks with Explicit Occlusion Training

Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos

Contact Info

Product

Resources

About