Master Motor Map (MMM) — Framework and toolkit for capturing, representing, and reproducing human motion on humanoid robots

Terlemez, Orner; Ulbrich, Stefan; Mandery, Christian; Do, Martin; Vahrenkamp, Nikolaus; Asfour, Tamim

doi:10.1109/humanoids.2014.7041470

Cited by 66 publications

(36 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most notably these are the Phobos project 5 and the RobotEditor of the OpenGrasp simulation suite (León et al, 2010; Terlemez et al, 2014). The RobotEditor project was finally chosen as the basis of the Robot Designer after an evaluation with competing projects.…”

Section: Software Architecturementioning

confidence: 99%

Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform

et al. 2017

Self Cite

View full text Add to dashboard Cite

Combined efforts in the fields of neuroscience, computer science, and biology allowed to design biologically realistic models of the brain based on spiking neural networks. For a proper validation of these models, an embodiment in a dynamic and rich sensory environment, where the model is exposed to a realistic sensory-motor task, is needed. Due to the complexity of these brain models that, at the current stage, cannot deal with real-time constraints, it is not possible to embed them into a real-world task. Rather, the embodiment has to be simulated as well. While adequate tools exist to simulate either complex neural networks or robots and their environments, there is so far no tool that allows to easily establish a communication between brain and body models. The Neurorobotics Platform is a new web-based environment that aims to fill this gap by offering scientists and technology developers a software infrastructure allowing them to connect brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems for in silico experimentation. In order to simplify the workflow and reduce the level of the required programming skills, the platform provides editors for the specification of experimental sequences and conditions, environments, robots, and brain–body connectors. In addition to that, a variety of existing robots and environments are provided. This work presents the architecture of the first release of the Neurorobotics Platform developed in subproject 10 “Neurorobotics” of the Human Brain Project (HBP).1 At the current state, the Neurorobotics Platform allows researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to simplified versions of brain models. We illustrate the capabilities of the platform with three example experiments: a Braitenberg task implemented on a mobile robot, a sensory-motor learning task based on a robotic controller, and a visual tracking embedding a retina model on the iCub humanoid robot. These use-cases allow to assess the applicability of the Neurorobotics Platform for robotic tasks as well as in neuroscientific experiments.

show abstract

Section: Software Architecturementioning

confidence: 99%

Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The framework includes a range of tools, amongst other 3D visualizer that can be used to inspect MMM motion data. Terlemez et al [27] discuss the data format and the available tools in depth.…”

Section: B Representation Of Human Motionmentioning

confidence: 99%

The KIT Motion-Language Dataset

2016

Self Cite

View full text Add to dashboard Cite

Linking human motion and natural language is of great interest for the generation of semantic representations of human activities as well as for the generation of robot activities based on natural language input. However, although there have been years of research in this area, no standardized and openly available data set exists to support the development and evaluation of such systems. We, therefore, propose the Karlsruhe Institute of Technology (KIT) Motion-Language Dataset, which is large, open, and extensible. We aggregate data from multiple motion capture databases and include them in our data set using a unified representation that is independent of the capture system or marker set, making it easy to work with the data regardless of its origin. To obtain motion annotations in natural language, we apply a crowd-sourcing approach and a web-based tool that was specifically build for this purpose, the Motion Annotation Tool. We thoroughly document the annotation process itself and discuss gamification methods that we used to keep annotators motivated. We further propose a novel method, perplexity-based selection, which systematically selects motions for further annotation that are either under-represented in our data set or that have erroneous annotations. We show that our method mitigates the two aforementioned problems and ensures a systematic annotation process. We provide an in-depth analysis of the structure and contents of our resulting data set, which, as of October 10, 2016, contains 3911 motions with a total duration of 11.23 hours and 6278 annotations in natural language that contain 52,903 words. We believe this makes our data set an excellent choice that enables more transparent and comparable research in this important area.

show abstract

“…Virtual markers that match the markers placed on the subject are then added to the MMM reference model at the corresponding locations. One converter is already provided by the MMM framework for the reconstruction of human motion from motion capture [1]. This converter uses a framewise gradientbased optimization approach based on the Jacobian matrix of the MMM kinematics to fit the model pose and is compared to the new approach proposed in this paper in Sec.…”

Section: The Master Motor Map Frameworkmentioning

confidence: 99%

“…The considered estimation task poses an additional challenge for Kalman filters: a (nonlinear) Kalman filter, by design, only estimates unconstrained quantities. That is, directly estimating θ k with a Kalman filter will violate the constraints (1). Recall that the estimate of a Kalman filter is represented by a mean vector and a covariance matrix.…”

Section: Satisfying the Joint Angle Bound Constraintsmentioning

confidence: 99%

Real-time whole-body human motion tracking based on unlabeled markers

Steinbring

Mandery

Pfaff

et al. 2016

2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)

Self Cite

View full text Add to dashboard Cite

We present a new online approach to track human whole-body motion from motion capture data, i.e., positions of labeled markers attached to the human body. Tracking in noisy data can be effectively performed with the aid of well-established recursive state estimation techniques. This allows us to systematically take noise of the marker measurements into account. However, as joint limits imposed by the human body have to be satisfied during estimation, first we transform this constrained estimation problem into an unconstrained one by using periodic functions. Then, we apply the Smart Sampling Kalman Filter to solve this unconstrained estimation problem. The proposed recursive state estimation approach makes the human motion tracking very robust to partial occlusion of markers and avoids any special treatment or reconstruction of the missed markers. A concrete implementation built on the kinematic human reference model of the Master Motor Map framework and a Vicon motion capture system is evaluated. Different captured motions show that our implementation can accurately estimate whole-body human motion in real-time and outperforms existing gradient-based approaches. In addition, we demonstrate its ability to smoothly handle incomplete marker data.

show abstract

Master Motor Map (MMM) — Framework and toolkit for capturing, representing, and reproducing human motion on humanoid robots

Cited by 66 publications

References 20 publications

Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform

Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform

The KIT Motion-Language Dataset

Real-time whole-body human motion tracking based on unlabeled markers

Contact Info

Product

Resources

About