Describing Upper-Body Motions Based on Labanotation for Learning-from-Observation Robots

Abstract

We have been developing a paradigm that we call learning-from-observation for a robot to automatically acquire a robot program to conduct a series of operations, or for a robot to understand what to do, through observing humans performing the same operations. Since a simple mimicking method to repeat exact joint angles or exact end-effector trajectories does not work well because of the kinematic and dynamic differences between a human and a robot, the proposed method employs intermediate symbolic representations, tasks, for conceptually representing what-to-do through observation. These tasks are subsequently mapped to appropriate robot operations depending on the robot hardware. In the present work, task models for upper-body operations of humanoid robots are presented, which are designed on the basis of Labanotation. Given a series of human operations, we first analyze the upper-body motions and extract certain fixed poses from key frames. These key poses are translated into tasks represented by Labanotation symbols. Then, a robot performs the operations corresponding to those task models. Because tasks based on Labanotation are independent of robot hardware, different robots can share the same observation module, and only different task-mapping modules specific to robot hardware are required. The system was implemented and demonstrated that three different robots can automatically mimic human upper-body operations with a satisfactory level of resemblance.

Task-Oriented Motion Mapping on Robots of Various Configuration Using Body Role Division

“…2) Fewer Degrees of Freedom: One approach to map arm postures to robots that have only one arm link is to sum the named pointing direction of the human upper arm and forearm into one direction [20]. However, while this approach may be suitable for gesture motion, manipulation motion have a slightly different characteristic.…”

“…In order to cover a variety of arm postures, we name a motion for every possible combination of the human upper and forearm pointing directions in some digitalized direction space (Figure 3). In this paper, we will use a direction space used in existing human motion representations [14] [15], where the direction is divided into eight horizontal directions (forward, left forward, left, ...) and five vertical directions (south pole, low, middle, high, north pole). To our survey, when limited to in-front single arm manipulations, the number of valid direction combinations of the human upper arm and forearm (the number of named postures) is 79 using this eight-by-five digitalization.…”

Robot Ego‐Noise Suppression with Labanotation‐Template Subtraction

“…Details of commanding a robot using Labanotation are explained in Ref. [14]. For these reasons, this study assumes the robot application is using the Labanotation-based commanding, and so we aim to use the noise measured at each Labanotation transition as a template for the NTS method.…”

“…This allows multi degree-of-freedom (DOF) robot motions to be confined to the limited motion space of human DOFs. From our previous survey [14], few methods have described long-term human-like robot motions. A method for preparing long templates is to use a set of predefined robot motions [13].…”

Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio

“…Synthesizing motions through learning techniques is becoming an increasingly popular approach to alleviating the requirement of capturing new real motion data to produce animations. The motion synthesis has been applied to a myriad of applications such as graphic animation for entertainment, robotics, and multimodal graphic rendering engines with human crowds [21], to name a few. Movements of each human being can be considered unique having its particularities, yet such movements preserve the characteristics of the motion style (e.g., walking, jumping, or dancing), and we are often capable of identifying the style effortlessly.…”