A Baseline for Cross-Database 3D Human Pose Estimation

Abstract

Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work.

A Step Forward Understanding Directional Limitations in Markerless Smartphone-based Gait Analysis

“…Max RMSE errors were also lower in WF. According to previous findings [18,29,30], there is a bias in pose estimation algorithms regarding the position and distances of the used cameras and currently available open-source training datasets that were never designed with biomechanical applications in mind. We can conclude that the errors observed in the markerless system related to directional differences are primarily due to less effective tracking when participants have their backs turned toward the iPhone cameras and walk away from the cameras.…”

Open-VICO: An Open-Source Gazebo Toolkit for Vision-based Skeleton Tracking in Human-Robot Collaboration

“…Open-VICO is the ideal environment to crosscompare and fuse the results of these systems offering the footprint of a bottleneck custom ROS message for joint position and name harmonization compatible with the human landmarks described in sectio II-B. In [26] there are some guidelines on how to perform this harmonization procedure among different training datasets (e.g., COCO) typically used in deep learning-based skeleton tracking systems. Open-VICO's structure allows the user to easily append additional tracking algorithms to the default list, enriching the possible comparison combinations.…”

Mobile Device-Based Video Screening for Infant Head Lag: An Exploratory Study

“…HPE algorithms sometimes output incorrect keypoint coordinates, and researchers are still working on solving this issue [34]. Although there are some open datasets for researchers to develop action-recognizing models, most of these datasets are used for learning everyday actions, such as handshaking, talking, standing, and sitting, and do not apply to infant head lag studies [42]. In addition to calling for more open datasets in this field, further studies could explore the potential of the generative adversarial network (GAN) to influence the technology-assisted diagnosis of developmental delays in infants [21].…”