Real-Time Whole-Body Imitation by Humanoid Robots and Task-Oriented Teleoperation Using an Analytical Mapping Method and Quantitative Evaluation

Abstract

Due to the limitations on the capabilities of current robots regarding task learning and performance, imitation is an efficient social learning approach that endows a robot with the ability to transmit and reproduce human postures, actions, behaviors, etc., as a human does. Stable whole-body imitation and task-oriented teleoperation via imitation are challenging issues. In this paper, a novel comprehensive and unrestricted real-time whole-body imitation system for humanoid robots is designed and developed. To map human motions to a robot, an analytical method called geometrical analysis based on link vectors and virtual joints (GA-LVVJ) is proposed. In addition, a real-time locomotion method is employed to realize a natural mode of operation. To achieve safe mode switching, a filter strategy is proposed. Then, two quantitative vector-set-based methods of similarity evaluation focusing on the whole body and local links, called the Whole-Body-Focused (WBF) method and the Local-Link-Focused (LLF) method, respectively, are proposed and compared. Two experiments conducted to verify the effectiveness of the proposed methods and system are reported. Specifically, the first experiment validates the good stability and similarity features of our system, and the second experiment verifies the effectiveness with which complicated tasks can be executed. At last, an imitation learning mechanism in which the joint angles of demonstrators are mapped by GA-LVVJ is presented and developed to extend the proposed system.

Experimental Testing of Bandstop Wave Filter to Mitigate Wave Reflections in Bilateral Teleoperation

“…The presence of e −2sT in Equations (15) and (16) enables the finite and infinite filters to mitigate the dominant frequency components of the wave reflections for limited cases, and it only depends on the time delay across the communication link. Finite and infinite filters are unsuitable when the frequency of wave reflection drift based on the changes to system dynamics of the teleoperated system [36].…”

“…Teleoperation involves controlling, operating, and manipulating remote robots and/or systems, in hostile environments, or in helping humans to accomplish strenuous tasks [1][2][3][4]. Since its conception, teleoperation has been applied in various areas such as space exploration [5], surveillance [6,7], volcano exploration, landmine detection [8], search and rescue [9], robotic surgery [10][11][12], mobile robots [13][14][15][16], and dealing with corrosive and deadly materials or substances [17]. The three main components of a teleoperation system are the master device, the remote or slave device, and the communication channel.…”

A Real-Time Upper-Body Robot Imitation System

“…Besides, mapping head motions provides a new way to control the orientation of the camera located on the robot's head so that the view of the robot changes following human's head, which contributes to monitoring the real-time environment around the robot. [18] Kinect SDK supports detecting and tracking the user's face. With inputs of color and depth images, the 3D head pose can be derived, the principle of which is presented in Refs.…”

Learning to gesticulate by observation using a deep generative approach

“…Motion capture (MoCap) is the process of recording motion data through any type of sensor. Applications of MoCap systems range from animation, bio-mechanics, medicine to sports science, entertainment, robotics [21] [31] or even study of animal behavior [27]. MoCap systems rely on optical technologies, and can be marker-based (e.g.…”