Human Movement Science in The Wild: Can Current Deep-Learning Based Pose Estimation Free Us from The Lab?

Abstract

Human movement researchers are often restricted to laboratory environments and data capture techniques that are time and/or resource intensive. Markerless pose estimation algorithms show great potential to facilitate large scale movement studies ‘in the wild’, i.e., outside of the constraints imposed by marker-based motion capture. However, the accuracy of such algorithms has not yet been fully evaluated. We computed 3D joint centre locations using several deep-learning based pose estimation methods (OpenPose, AlphaPose, DeepLabCut) and compared to marker-based motion capture. Participants performed walking, running and jumping activities while marker-based motion capture data and multi-camera high speed images (200 Hz) were captured. The pose estimation algorithms were applied to 2D image data and 3D joint centre locations were reconstructed. Pose estimation derived joint centres demonstrated systematic differences at the hip and knee (~30 − 50 mm), most likely due to mislabeling of ground truth data in the training datasets. Where systematic differences were lower, e.g., the ankle, differences of 1 − 15 mm were observed depending on the activity. Markerless motion capture represents a highly promising emerging technology that could free movement scientists from laboratory environments. We provide recommendations relating to domain specific datasets and benchmarks, which will be vital to realising this goal.

Review—Emerging Portable Technologies for Gait Analysis in Neurological Disorders

“…Although not compared to a marker-based system, DeepLabCut seemed sensitive enough to differentiate between closely-spaced running cadences with a high test-retest reliability of the mean stride data. In 3D motion capture obtained with OpenPose, DeepLabCut, and AlphaPose, significant kinematic differences at hip and knee occurred in comparison to marker-based systems (Needham et al, 2021). Here, tracking accuracy of the ankle unexpectedly performed better than other joints, possibly owing to more precise manual annotation during training due to its apparent anatomical position.…”

Modeling Physical Activity in Children by Combining Raw Hip-Worn Accelerometry, 2D Pose Estimation, and Direct Observation

“…Two-dimensional human pose estimation [15] entails locating and associating the joints of humans in digital images thus forming a 2D projection of the body pose (kinematic skeleton). The potential value of these methods for movement science is already being noticed as it could eliminate the restrictions of marker-based motion capture [16]. Our goal is to find out whether the detected poses as relatively objective representations of the human body in video can be used for modeling PA intensity.…”