Multi-Person 3D Human Pose Estimation from Monocular Images

Dabral, Rishabh; Gundavarapu, Nitesh B.; Mitra, Rahul; Sharma, Abhishek; Ramakrishnan, Ganesh; Jain, Arjun

doi:10.1109/3dv.2019.00052

Cited by 56 publications

(38 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To that point, 3D models may prove more feature computation more robust to the angle of recording, and so it is possible that 3D reconstruction would improve the performance of this system. Recent studies have shown promise in estimating full 3D pose reconstruction based on data recorded using a single 2D camera [ 63 , 64 , 65 , 66 ], although challenges remain [ 21 ].…”

Section: Discussionmentioning

confidence: 99%

A Clinically Interpretable Computer-Vision Based Method for Quantifying Gait in Parkinson’s Disease

Rupprechter¹,

Morinan²,

Peng³

et al. 2021

Sensors

View full text Add to dashboard Cite

Gait is a core motor function and is impaired in numerous neurological diseases, including Parkinson’s disease (PD). Treatment changes in PD are frequently driven by gait assessments in the clinic, commonly rated as part of the Movement Disorder Society (MDS) Unified PD Rating Scale (UPDRS) assessment (item 3.10). We proposed and evaluated a novel approach for estimating severity of gait impairment in Parkinson’s disease using a computer vision-based methodology. The system we developed can be used to obtain an estimate for a rating to catch potential errors, or to gain an initial rating in the absence of a trained clinician—for example, during remote home assessments. Videos (n=729) were collected as part of routine MDS-UPDRS gait assessments of Parkinson’s patients, and a deep learning library was used to extract body key-point coordinates for each frame. Data were recorded at five clinical sites using commercially available mobile phones or tablets, and had an associated severity rating from a trained clinician. Six features were calculated from time-series signals of the extracted key-points. These features characterized key aspects of the movement including speed (step frequency, estimated using a novel Gamma-Poisson Bayesian model), arm swing, postural control and smoothness (or roughness) of movement. An ordinal random forest classification model (with one class for each of the possible ratings) was trained and evaluated using 10-fold cross validation. Step frequency point estimates from the Bayesian model were highly correlated with manually labelled step frequencies of 606 video clips showing patients walking towards or away from the camera (Pearson’s r=0.80, p<0.001). Our classifier achieved a balanced accuracy of 50% (chance = 25%). Estimated UPDRS ratings were within one of the clinicians’ ratings in 95% of cases. There was a significant correlation between clinician labels and model estimates (Spearman’s ρ=0.52, p<0.001). We show how the interpretability of the feature values could be used by clinicians to support their decision-making and provide insight into the model’s objective UPDRS rating estimation. The severity of gait impairment in Parkinson’s disease can be estimated using a single patient video, recorded using a consumer mobile device and within standard clinical settings; i.e., videos were recorded in various hospital hallways and offices rather than gait laboratories. This approach can support clinicians during routine assessments by providing an objective rating (or second opinion), and has the potential to be used for remote home assessments, which would allow for more frequent monitoring.

show abstract

Section: Discussionmentioning

confidence: 99%

A Clinically Interpretable Computer-Vision Based Method for Quantifying Gait in Parkinson’s Disease

Rupprechter¹,

Morinan²,

Peng³

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Several further algorithms use different variants of anatomical constraints for the human body (e.g., body symmetry) and show improved results in weakly-supervised [Dabral et al 2018;Wandt and Rosenhahn 2019] or even unsupervised 3D human pose estimation [Kovalenko et al 2019]. [Hassan et al 2019;] use geometric vicinity and collision avoidance constraints for the reconstruction of human-object interactions, and [Dabral et al 2019;Fabbri et al 2020;Mehta et al 2020;Rogez et al 2019;Zanfir et al 2018] can generalise to multiple persons in the scene.…”

Section: Related Workmentioning

confidence: 99%

Neural monocular 3D human motion capture with physical awareness

Shimada

Golyanik

et al. 2021

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…Some preliminary works [11], [12] have proposed methods to recover a complete 3D pose from RGB images with promising results. However, even if these methods predict a good estimation of the pose, they fail to recover the correct positioning in the camera space as well as the real scale of the body [12]. Thus, the errors will affect the computation of the corresponding measures of body parts and limbs (e.g.…”

Section: Introductionmentioning

confidence: 99%

RefiNet: 3D Human Pose Refinement with Depth Maps

D’Eusanio

Pini

Borghi

et al. 2021

2020 25th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Human Pose Estimation is a fundamental task for many applications in the Computer Vision community and it has been widely investigated in the 2D domain, i.e. intensity images. Therefore, most of the available methods for this task are mainly based on 2D Convolutional Neural Networks and huge manuallyannotated RGB datasets, achieving stunning results. In this paper, we propose RefiNet, a multi-stage framework that regresses an extremely-precise 3D human pose estimation from a given 2D pose and a depth map. The framework consists of three different modules, each one specialized in a particular refinement and data representation, i.e. depth patches, 3D skeleton and point clouds. Moreover, we present a new dataset, called Baracca, acquired with RGB, depth and thermal cameras and specifically created for the automotive context. Experimental results confirm the quality of the refinement procedure that largely improves the human pose estimations of off-the-shelf 2D methods.

show abstract

Multi-Person 3D Human Pose Estimation from Monocular Images

Cited by 56 publications

References 41 publications

A Clinically Interpretable Computer-Vision Based Method for Quantifying Gait in Parkinson’s Disease

A Clinically Interpretable Computer-Vision Based Method for Quantifying Gait in Parkinson’s Disease

Neural monocular 3D human motion capture with physical awareness

RefiNet: 3D Human Pose Refinement with Depth Maps

Contact Info

Product

Resources

About