Application of extended Kalman filter for improving the accuracy and smoothness of Kinect skeleton-joint estimates

Shu, Jody; Hamano, Fumio; Angus, John E.

doi:10.1007/s10665-014-9689-2

Cited by 33 publications

(14 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, non-linear KFs provide another idea to solve the low-accuracy problems in D-Mocap. In [13], Researchers use the sound localization capabilities of the Kinect sensor and 2 Le Zhou et al…”

Section: Related Workmentioning

confidence: 99%

“…However, human motion is a nonlinear system especially in the case of D-Mocap which suffers from distortion and outliers. Researchers have explored the use of the extended KF (EKF) [13] and the unscented KF (UKF) [14] in order to compensate for the nonlinear nature of D-Mocap data. However, these nonlinear methods are still limited to handle the non-Gaussian and biased noise inherent in the D-Mocap data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Zhou¹,

Lannan²,

Fan³

et al. 2021

EAI Endorsed Transactions on Bioengineering and Bioinformatics

View full text Add to dashboard Cite

INTRODUCTION:Recently, low cost RGB-D depth sensors have emerged as a promising alternative for motion capture for clinical gait assessment. However, depth sensor based Mocap (D-Mocap) suffers from low accuracy and poor stability for 3D joint estimation due to noise, self-occlusion, interference, and other technical limitations, which prevents it from being widely used in health-related applications. OBJECTIVES: The primary objective of this study is to integrate non-linear Kalman filters (KFs) and evolutionary algorithms to enhance the quality of D-Mocap data by jointly considering kinematic or anthropometric constraints at the joint-level and skeleton-level, respectively. METHODS: We propose a hybrid approach to synergistically integrate the Tobit Kalman filter (TKF) and the Differential Evolutionary (DE) algorithm for human motion enhancement that is referred to as TKF-DE. Specifically, the joint-level TKF provides the predictive distribution of each joint that is kinematically admissible in time and probabilistically amenable in space for skeleton-level DE optimization in terms of all bone lengths. Two predictive distributions of the TKF, i.e., the Gaussian and uniform, are tested and compared in terms of their effectiveness in generating the initial DE population. RESULTS: Two sets of motion capture data are used to validate the proposed TKF-DE methods, one simulated and one real-world. The first dataset is from the Carnegie Mellon University Database (CMU) which contains a multitude of various motions and is simplified to a 21-joint skeleton and corrupted with additive white Gaussian noise (AWGN). The second dataset was collected at two labs at Oklahoma State University (OSU). The Orbbec depth sensor and the Nuitrack SDK were used for D-Mocap data acquisition along with an optical Mocap system that was time-synchronized and skeleton-matched with the D-Mocap system as a reference for evaluation. The results confirm that the proposed TKF-DE algorithms significantly outperform other nonlinear KFs, including the extended KF (EKF), the unscented KF (UKF), and the TKF alone, with improved accuracy and stability of the estimation of joint positions, bone lengths, and joint angles. It is also shown that the Gaussian-based predictive distribution is better than the uniform one, further validating the efficacy and synergy of the two key components in the TKF-DE algorithm. CONCLUSION: Our research synergistically integrates the TKF and DE algorithms in one framework referred to as the TKF-DE, that takes advantage of kinematic and anthropometric constraints for human motion enhancement. The experimental results on two D-Mocap datasets show that the proposed TKF-DE method significantly improves the quality of D-Mocap data in terms of joint positions, bone lengths, and joint angles. This study takes one step closer to bringing the D-Mocap technology to possible health-related applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Zhou¹,

Lannan²,

Fan³

et al. 2021

EAI Endorsed Transactions on Bioengineering and Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Unfortunately, they did not consider the use of this algorithm for joint angle estimates. Shu et al [26] used an extended Kalman filter (EKF) to improve kinematic estimates from pose data. Unfortunately, their algorithm was only implemented with the head joints, but they were able to obtain an accuracy of 0.039 m on the head JCP with Kalman filtering.…”

Section: Related Workmentioning

confidence: 99%

Physically Consistent Whole-Body Kinematics Assessment Based on an RGB-D Sensor. Application to Simple Rehabilitation Exercises

Colombel

Bonnet

Daney

et al. 2020

Sensors

View full text Add to dashboard Cite

This work proposes to improve the accuracy of joint angle estimates obtained from an RGB-D sensor. It is based on a constrained extended Kalman Filter that tracks inputted measured joint centers. Since the proposed approach uses a biomechanical model, it allows physically consistent constrained joint angles and constant segment lengths to be obtained. A practical method that is not sensor-specific for the optimal tuning of the extended Kalman filter covariance matrices is provided. It uses reference data obtained from a stereophotogrammetric system but it has to be tuned only once since it is task-specific only. The improvement of the optimal tuning over classical methods in setting the covariance matrices is shown with a statistical parametric mapping analysis. The proposed approach was tested with six healthy subjects who performed four rehabilitation tasks. The accuracy of joint angle estimates was assessed with a reference stereophotogrammetric system. Even if some joint angles, such as the internal/external rotations, were not well estimated, the proposed optimized algorithm reached a satisfactory average root mean square difference of 9.7 ∘ and a correlation coefficient of 0.8 for all joints. Our results show that an affordable RGB-D sensor can be used for simple in-home rehabilitation when using a constrained biomechanical model.

show abstract

“…Further improvements have been achieved by additionally enforcing kinematic constraints [16], using a multi-channel mixture of parts model with kinematic constraints to improve the robustness of the estimation of the joint positions [17], or encountering the high level of jitter due to noise and estimation errors by applying an extended Kalman filter and the exploitation of sound cues [18]. Furthermore, based on a setup with two Kinects, Yeung et al [19] approach the human skeleton estimation based on a constrained optimization framework that penalized deviations of the 3-D joint location hypotheses provided by the Kinect SDK for the individual Kinects and that enforced constant bone lengths.…”

Section: Related Workmentioning

confidence: 99%

Constraint-Based Optimized Human Skeleton Extraction from Single-Depth Camera

Weinmann

et al. 2019

Sensors

View full text Add to dashboard Cite

As a cutting-edge research topic in computer vision and graphics for decades, human skeleton extraction from single-depth camera remains challenging due to possibly occurring occlusions of different body parts, huge appearance variations, and sensor noise. In this paper, we propose to incorporate human skeleton length conservation and symmetry priors as well as temporal constraints to enhance the consistency and continuity for the estimated skeleton of a moving human body. Given an initial estimation of the skeleton joint positions provided per frame by the Kinect SDK or Nuitrack SDK, which do not follow the aforementioned priors and can prone to errors, our framework improves the accuracy of these pose estimates based on the length and symmetry constraints. In addition, our method is device-independent and can be integrated into skeleton extraction SDKs for refinement, allowing the detection of outliers within the initial joint location estimates and predicting new joint location estimates following the temporal observations. The experimental results demonstrate the effectiveness and robustness of our approach in several cases.

show abstract

Application of extended Kalman filter for improving the accuracy and smoothness of Kinect skeleton-joint estimates

Cited by 33 publications

References 9 publications

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Human Motion Enhancement via Joint Optimization of Kinematic and Anthropometric Constraints

Physically Consistent Whole-Body Kinematics Assessment Based on an RGB-D Sensor. Application to Simple Rehabilitation Exercises

Constraint-Based Optimized Human Skeleton Extraction from Single-Depth Camera

Contact Info

Product

Resources

About