Augmented reality environment with virtual fixtures for robotic telemanipulation in space

Abstract

This paper presents an augmented reality framework, implemented on the master console of a modified da Vinci R surgical robot, that enables the operator to design and implement assistive virtual fixtures during teleoperation. Our specific goal is to facilitate teleoperation with large time delays, such as the delay of several seconds that occurs with groundbased control of robotic systems in earth orbit. The virtual fixtures give immediate visual feedback and motion guidance to the operator, while the remote slave performs motions consistent with those constraints. This approach is suitable for tasks in unstructured environments, such as servicing of existing on-orbit spacecraft that were not designed for servicing. We conducted a pilot study by teleoperating a remote slave robot for a thermal barrier blanket cutting task using virtual fixtures with and without time delay. The results show that virtual fixtures reduce the time required to complete the task while also eliminating significant manipulation errors, such as tearing the blanket. The improvement in performance is especially dramatic when a simulated time delay (4 seconds) is introduced.

Registration of planar virtual fixtures by using augmented reality with dynamic textures

“…At this short range, the MLI surface fills the entire field of view and little or no visual cues are available unless the edges of the of plate are visible. This stereo configuration is used on the slave side of the JHU satellite servicing testbench [15] illustrated in Fig. 9.…”

“…Although access to 3D data can be used to model the geometry of fixtures, methods that use structured light cannot be applied to shinny surfaces. A human-computer interaction (HCI) environment was presented in [15] to manually adjust the virtual fixtures but the alignment was done in a virtual environment and no sensing data was provided to assess the alignment.…”

Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing

“…The distortion parameters are directly obtained from the Kinect camera, through the openni camera ROS stack. The second module provides virtual fixtures [Rosenberg 1993] which have been proven to be very helpful in teleoperation tasks [Xia et al 2012]. Virtual representation of the objects are displayed according to the information received by the object recognition node: the color of the virtual object is changed according to a RGB color scale following the distance between the robot end-effector and the object position.…”

Task frame estimation during model-based teleoperation for satellite servicing

“…The robot is then oriented to align the cutter axis, z c (shown in Figure 1), with the normal of the surface, n p , underlying the patch. Hybrid position/force control [2] is used to keep the cutting tool in contact with the surface while the user teleoperates along the tangent plane to perform the cutting task [3], [4], [5]. Specifically, force control is applied along z c while the user teleoperates in position/velocity control along x c and y c , possibly constrained by virtual fixtures (e.g., to limit motion along y c [6]).…”