A Novel Evaluation Framework for Teleoperation and a Case Study on Natural Human-Arm-Imitation Through Motion Capture

Abstract

Although tele-operation has a long history, when it comes to tuning, comparison, and evaluation of teleoperation systems, no standard framework exists which can fulfill desiderata such as: concisely modeling multiple aspects of the system as a whole, i.e. timing, accuracy, and event transitions, while also providing for separation of user-, feedback-, as well as learning-dependent components. On the other hand, real-time remote tele-operation of robotic arms, either industrial or humanoid, is highly suitable for a number of applications, especially in difficult or inaccessible environment, and thus such an evaluation framework would be desirable. Usually, teleoperation is driven by buttons, joysticks, haptic controllers, or slave-arms, providing an interface which can be quite cumbersome and unnatural, especially when operating robots with multiple degrees of freedom. Thus, in thus paper, we present a twofold contribution: (a) a task-based teleoperation evaluation framework which can achieve the desiderata described above, as well as (b) a system for teleoperation of an industrial arm commanded through human-arm motion capture, which is used as a case study, and also serves to illustrate the effectiveness of the evaluation framework that we are introducing. In our system the desired trajectory of a remote robotic arm is easily and naturally controlled through imitation of simple movements of the operator's physical arm, obtained through motion capture. Furthermore, an extensive real-world evaluation is provided, based on our proposed probabilistic framework, which contains an intersubject quantitative study with 23 subjects, a longitudinal

Performance and Usability Evaluation Scheme for Mobile Manipulator Teleoperation

“…A task-based evaluation framework for teleoperation is presented in [20]. The evaluation framework comprises a taskbased measurable parameter based on successful and unsuccessful movements, and user-opinion data are obtained through a questionnaire.…”

Predicting Human-Robot Team Performance Based on Cognitive Fatigue

“…Some research focuses on evaluating the robot itself [8], while others concentrate on the interface side [9], [10]. Additionally, certain studies have attempted to incorporate human factors into the overall system performance evaluation [3], [4], [11], [12].…”

“…One study proposes a task-based framework, which considers successful and unsuccessful motions along with user opinions [11]. Other evaluation methods analyse the workload on participants in robot teleoperation using the NASA Task Load Index (NASA-TLX) and other customised scales [3], [4].…”

Semi-Autonomous Domestic Service Robots: Evaluation of a User Interface for Remote Manipulation and Navigation With Focus on Effects of Stereoscopic Display

“…For example, fixed viewpoints on the remote scene [27,28], misalignments between the user's viewpoint and the coordinate system for controlling the end effector [29], and restrictions on the degrees of freedom (DOF) for controlling the end effector [30] have shown to be detrimental for task performance. Direct unassisted telemanipulation is often associated with problems like environment collisions due to transmission delay [31,32] or the need for sophisticated and expensive interaction devices to mirror the robot's dexterous abilities [33][34][35].…”