The KIT whole-body human motion database

Mandery, Christian; Terlemez, Ömer; Do, Martin; Vahrenkamp, Nikolaus; Asfour, Tamim

doi:10.1109/icar.2015.7251476

Cited by 174 publications

(114 citation statements)

References 11 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…There are many motion capture datasets [42,14,31,32,38,8,41,44] as well as several attempts to aggregate such datasets into larger collections [3,21,29]. Previous attempts to merge datasets [21,29] adopt a common body representation in which the size variation among subjects is normalized. This enables methods that focus on modeling pose and motion in terms of joint locations.…”

Section: Related Workmentioning

confidence: 99%

AMASS: Archive of Motion Capture As Surface Shapes

Mahmood¹,

Ghorbani

Troje

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

881

481

View full text Add to dashboard Cite

Large datasets are the cornerstone of recent advances in computer vision using deep learning. In contrast, existing human motion capture (mocap) datasets are small and the motions limited, hampering progress on learning models of human motion. While there are many different datasets available, they each use a different parameterization of the body, making it difficult to integrate them into a single meta dataset. To address this, we introduce AMASS, a large and varied database of human motion that unifies 15 different optical marker-based mocap datasets by representing them within a common framework and parameterization. We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model; here we use SMPL [28], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh. The method works for arbitrary marker sets, while recovering soft-tissue dynamics and realistic hand motion. We evaluate MoSh++ and tune its hyperparameters using a new dataset of 4D body scans that are jointly recorded with markerbased mocap. The consistent representation of AMASS makes it readily useful for animation, visualization, and generating training data for deep learning. Our dataset is significantly richer than previous human motion collections, having more than 40 hours of motion data, spanning over 300 subjects, more than 11000 motions, and will be publicly available to the research community. Figure 1: We unify a large corpus of archival marker-based optical human mocap datasets by representing them within a common framework and parameterization. A sampling of shapes and poses from a few datasets in AMASS is shown, from left to right: CMU [14], MPI-HDM05 [32, 31], MPI-Pose Limits [8], KIT [29], BioMotion Lab [41], TCD [22]and ACCAD [5] datasets. The input is sparse markers and the output is SMPL body models.

show abstract

Section: Related Workmentioning

confidence: 99%

AMASS: Archive of Motion Capture As Surface Shapes

Mahmood¹,

Ghorbani

Troje

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

881

481

View full text Add to dashboard Cite

show abstract

“…Sometimes the faces of patients are clinical manifestation of movement disorder [2]. An alternative for sharing patient movement videos is to extract 3D keypoints and publish the arXiv:2003.00813v1 [cs.CV] 7 Feb 2020 keypoints data instead [1,20]. However, raw videos could provide far richer information than merely keypoints.…”

Section: Introductionmentioning

confidence: 99%

Deepfakes for Medical Video De-Identification

Zhu

Fang

Sui

et al. 2020

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society

View full text Add to dashboard Cite

Data sharing for medical research has been difficult as open-sourcing clinical data may violate patient privacy. Traditional methods for face de-identification wipe out facial information entirely, making it impossible to analyze facial behavior. Recent advancements on whole-body keypoints detection also rely on facial input to estimate body keypoints. Both facial and body keypoints are critical in some medical diagnoses, and keypoints invariability after deidentification is of great importance. Here, we propose a solution using deepfake technology, the face swapping technique. While this swapping method has been criticized for invading privacy and portraiture right, it could conversely protect privacy in medical video: patients' faces could be swapped to a proper target face and become unrecognizable. However, it remained an open question that to what extent the swapping de-identification method could affect the automatic detection of body keypoints. In this study, we apply deepfake technology to Parkinson's disease examination videos to de-identify subjects, and quantitatively show that: faceswapping as a de-identification approach is reliable, and it keeps the keypoints almost invariant, significantly better than traditional methods. This study proposes a pipeline for video de-identification and keypoint preservation, clearing up some ethical restrictions for medical data sharing. This work could make open-source high quality medical video datasets more feasible and promote future medical research that benefits our society. CCS CONCEPTS• Security and privacy → Privacy protections.

show abstract

“…However, with the small dataset, unsupervised approaches perform very well. Concerning future work, the first step will be to investigate the performance of FADE with other motion databases, such as the Berkeley MHAD database [24], CMU Graphics Lab Motion Capture Database [27], and KIT Whole-Body Human Motion Database [28]. In this work, FADE has been applied to joint angles.…”

Section: Discussionmentioning

confidence: 99%

Encoding human actions with a frequency domain approach

Shah

Falco

Saveriano

et al. 2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Abstract-In this work, we propose a Frequency-based Action Descriptor (FADE) to represent human actions. In robotics, with the development of Programming by Demonstration (PbD) methods, representing and recognizing large sets of actions has become crucial to build autonomous systems that learn from humans. The FADE descriptor leverages Fast Fourier Transform (FFT) for action representation and is combined with the Manhattan distance for measuring similarities between actions. It is characterized by a low time and space complexity and is particularly suitable for classification of human actions. For clustering problems, we propose a modified version of FADE, called Uncompressed-FADE (U-FADE), which performs well in combination with Spectral Clustering algorithms at the price of a reduced compression. We compare FADE with action descriptors based on Singular Value Decomposition (SVD) and Hidden Markov Models (HMM) on the entire HDM05 motion capture database. Despite the high dimensionality of the problem, we obtained on the entire database a promising recognition rate of 78% combining FADE with a simple 1-NN classification algorithm. Furthermore, we achieved a rate of 98% on a small action set and 88% on a medium action set.

show abstract

The KIT whole-body human motion database

Cited by 174 publications

References 11 publications

AMASS: Archive of Motion Capture As Surface Shapes

AMASS: Archive of Motion Capture As Surface Shapes

Deepfakes for Medical Video De-Identification

Encoding human actions with a frequency domain approach

Contact Info

Product

Resources

About