Physically Consistent Whole-Body Kinematics Assessment Based on an RGB-D Sensor. Application to Simple Rehabilitation Exercises

Abstract

This work proposes to improve the accuracy of joint angle estimates obtained from an RGB-D sensor. It is based on a constrained extended Kalman Filter that tracks inputted measured joint centers. Since the proposed approach uses a biomechanical model, it allows physically consistent constrained joint angles and constant segment lengths to be obtained. A practical method that is not sensor-specific for the optimal tuning of the extended Kalman filter covariance matrices is provided. It uses reference data obtained from a stereophotogrammetric system but it has to be tuned only once since it is task-specific only. The improvement of the optimal tuning over classical methods in setting the covariance matrices is shown with a statistical parametric mapping analysis. The proposed approach was tested with six healthy subjects who performed four rehabilitation tasks. The accuracy of joint angle estimates was assessed with a reference stereophotogrammetric system. Even if some joint angles, such as the internal/external rotations, were not well estimated, the proposed optimized algorithm reached a satisfactory average root mean square difference of 9.7 ∘ and a correlation coefficient of 0.8 for all joints. Our results show that an affordable RGB-D sensor can be used for simple in-home rehabilitation when using a constrained biomechanical model.

Sparse Visual-Inertial Measurement Units Placement for Gait Kinematics Assessment

“…Cerveri et al [41] were the sole authors to propose a frequency based approach but this requires to have a prior knowledge of the joint angle evolution, applies to cyclic motions and was not tested with low-cost sensors data. We have shown in a previous study [16], using a RGB-D camera and a markerless joint center estimate algorithm, that an optimization process outperforms frequency based method and is absolutely required to tune the EKF parameters. Nevertheless, when using markerless data as input and despite an optimal tuning of the EKF parameters and a simplified experimental setup [16], it was not possible to achieve a joint estimate RMSD lower than 9.7deg on average when compared to a reference SS.…”

“…We have shown in a previous study [16], using a RGB-D camera and a markerless joint center estimate algorithm, that an optimization process outperforms frequency based method and is absolutely required to tune the EKF parameters. Nevertheless, when using markerless data as input and despite an optimal tuning of the EKF parameters and a simplified experimental setup [16], it was not possible to achieve a joint estimate RMSD lower than 9.7deg on average when compared to a reference SS. Specifically, for tasks closely related to walking, a RMSD up to 20deg was observed for the hip joints.…”

“…Devices, based on RGB-Depth sensors such as the Kinect camera (Microsoft), with embedded skeleton tracking algorithms are available. However, it was shown that the Kinect camera is not accurate enough for rehabilitation assessment mainly due to segment length variations [16]. For gait analysis, the Kinect sensor can only estimate timing and spatial characteristics [17] as joint angles are not reliable enough for a consistent analysis.…”

“…Numerous methods have been developed in the literature to improve kinematic estimates of RGB-D like sensors. Most of them add the use of a multi-body model to obtain a more robust joint angle estimate [15], [16]. However, acceptable differences can only be obtained for motions of very large amplitude [16].…”

Markerless 3D Human Pose Tracking in the Wild with Fusion of Multiple Depth Cameras: Comparative Experimental Study with Kinect 2 and 3

“…depth camera and wearable sensors as IMU) [4]. Especially for data from depth sensors, this type of algorithm can be also used with anthropometric constraints and a dynamical model for human motion to compensate for the lack of a model in the body tracking software [5]. One objective of this paper is to evaluate if this approach remains useful and meaningful for the body tracking software Kinect 3 (Azure Kinect).…”