Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Abstract

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges.

Whole-Body Control and Teleoperation of a Suspended Aerial Manipulator

“…Some of the results presented in this chapter were also published in Coelho et al (2019), Coelho et al (2020), Coelho et al (2021b), Hulin et al (2021), and Lee et al (2023).…”

“…Since its first appearance multiple applications and improvements to the TDPA method have been presented. Applications include high-delay space teleoperation (Artigas et al, 2016;Panzirsch et al, 2022), model-augmented teleoperation (Hulin et al, 2021), the proxy-based approach (Singh et al, 2020), force control (Balachandran et al, 2020), and cooperative control for autonomous landing (Muskardin et al, 2020). Due to its high range of applications and its demonstrated performance and robustness, TDPA appears to be a reasonable candidate to be applied in the bilateral teleoperation of aerial robots.…”

Towards a VR Evaluation Suite for Tactile Displays in Telerobotic Space Missions

“…Our ViESTac suite is a first step towards an evaluation tool to investigate tactile perception and the suitability of devices under microgravity conditions in future telerobotic space missions. Such a suite would also be important to evaluate novel telerobotic approaches and methods that aim to integrate local and remote models to facilitate task performance, as described in [48].…”

A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training

“…Switching between VFs may cause instabilities. While in practice instabilities aren't observed, a time domain passivity controller as introduced in Hannaford and Ryu (2002) and further developed in Hertkorn et al (2010), Ryu et al (2005, and Hulin et al (2021) is to be integrated into the presented approach in the future, to guarantee a safe operation for arbitrary tasks in all circumstances. Switching between VFs and their parametrization allows the flexible application of various VFs depending on the required task.…”