Joystick mapped Augmented Reality Cues for End-Effector controlled Tele-operated Robots

Abstract

End-effector control of robots using just remote camera views is difficult due to lack of perceived correspondence between the joysticks and the end-effector coordinate frame. This paper reports the positive effects of Augmented Reality visual cues on operator performance during end-effector controlled teleoperation using only camera views. Our solution is to overlay a color-coded coordinate system on the end-effector of the robot using AR techniques. This mapped and color-coded coordinate system is then directly mapped to similarly color-coded joysticks used for control of both position and orientation. The AR view along with mapped markings on the joystick give the user a clear notion of the effect of their joystick movements on the endeffector of the robot. All camera views display this registered dynamic overlay information on-demand. A preliminary test using fifteen subjects comparing control of performance with and without the coordinate mapping was performed by using a simple insertion task. Preliminary results indicate a significant reduction in distance, reversal errors and mental workload.

Modelling and control of a 2-link mobile manipulator with virtual prototyping

“…Sometimes it may not be efficient or feasible to construct a perfect the environment model online due to complex and fast changing situations, especially for some military and civil defense applications like battle field and rescue robots. Tele-operations are thereby sometimes more preferred for the sake of faster online task execution with human aid as well as more flexible interaction with the environment [7], [8], [9]. As a matter of fact, the robots are frequently requested to operate on rough and uneven terrains in such situations, which usually generate uncertainty to the robots knowledge about the environment.…”

Evaluation of a method for intuitive telemanipulation based on view-dependent mapping and inhibition of movements

“…the manipulator's coordinate system). A fixed mapping together with the option to change between different cameras or viewpoints may result in rather unexpected behavior or can at least be considered error-prone since the user is responsible of taking account of the implicit coordinate transformations [13].…”

Systematic Tele-operation with Augmented Reality Path Planned Navigation Cues in Cluttered Environments

“…We recently reported how AR cues improve operator teleoperation performance under conditions of display-control misalignment (Chintamani, Cao, Ellis, & Pandya, 2008 ;Nawab, Chintamani, Ellis, Auner, & Pandya, 2007). The technique investigated was simple: By registering pose estimated, color-coded graphic coordinate axes to the EE on the display, and placing similar color codes at the HCs ( Figure 1), operators are provided with a simple technique to map HC axes to the EE axes irrespective of viewport (visual-motor mapping) or EE misalignment (display-control misalignment).…”