Advanced teleoperation and control system for industrial robots based on augmented virtuality and haptic feedback

Abstract

There are some industrial tasks that are still mainly performed manually by human workers due to their complexity, which is the case of surface treatment operations (such as sanding, deburring, finishing, grinding, polishing, etc.) used to repair defects. This work develops an advanced teleoperation and control system for industrial robots in order to assist the human operator to perform the mentioned tasks. On the one hand, the controlled robotic system provides strength and accuracy, holding the tool, keeping the right tool orientation and guaranteeing a smooth approach to the workpiece. On the other hand, the advanced teleoperation provides security and comfort to the user when performing the task. In particular, the proposed teleoperation uses augmented virtuality (i.e., a virtual world that includes non-modeled real-world data) and haptic feedback to provide the user an immersive virtual experience when remotely teleoperating the tool of the robot system to treat arbitrary regions of the workpiece surface. The method is illustrated with a car body surface treatment operation, although it can be easily extended to other surface treatment applications or even to other industrial tasks where the human operator may benefit from robotic assistance. The effectiveness of the proposed approach is shown with several experiments using a 6R robotic arm.

Still Not Solved: A Call for Renewed Focus on User-Centered Teleoperation Interfaces

“…Some industries have well-defined and structured tasks and environments and tasks that could be leveraged as a simpler environment to develop better interfaces. Like more difficult applications of robots, these applications require robot navigation in constrained spaces, detailed inspection with multiple advanced sensors, logging, and reporting, and sometimes engaging in repair work using carefully controlled manipulators ( Buonocore, 2021 ; González et al, 2021 ; Koh et al, 2021 ). These operators are often specialized in the industry the robot is used in, but not necessarily familiar with robotics or similar technology themselves.…”

“…Recent calls for bringing in advanced information visualization techniques acknowledge this, and is an important future approach to improving teleoperation ( Szafir and Szafir, 2021 ). Some of these techniques even leverage new display technologies like augmented and virtual reality to explore new ways to present interfaces in a natural way ( Nielsen et al, 2007 ; Labonte et al, 2010 ; Hedayati et al, 2018 ; González et al, 2021 ). We would encourage additional focus on visualizations that consider less experienced users, who may not understand or be familiar with the limitations of sensor or data types, who will likely be slower with any technique, and may be harder to train due to lack of foundational knowledge in engineering, or computer science.…”

“…Teleoperated robots, robots controlled at a distance, are already used in a number of situations where it is unsafe for humans to be physically present, such as search-and-rescue ( Casper and Murphy, 2003 ; BBCClick, 2017 ; Peskoe-Yang, 2019 ) after natural disasters ( Settimi et al, 2014 ; Norton et al, 2017 ; Tadokoro, 2019 ), scientific research such as moving underwater ( Delmerico et al, 2019 ), working in space ( Schilling et al, 1997 ), or making critical inspection and repairs ( Buonocore, 2021 ; González et al, 2021 ; Koh et al, 2021 ). However, with a small number of exceptions, we see no remotely operated robots in use by the general public, whether it is at work or in their personal space.…”

Robust Control of a New Asymmetric Teleoperation Robot Based on a State Observer

“…If the speed of the end effector from the slave robot is not measurable, the state observer can be considered. However, due to the nonlinear uncertainty of the plant and that the close-loop system must satisfy the gain, the observer cannot be designed based on the separation principle [ 36 ]. We re-examine the performance of the observer based on the close-loop system.…”

Hand Guiding a Virtual Robot Using a Force Sensor

“…FT sensors can be used effectively in various applications with industrial robots, but they are also often used as safety sensors. This is demonstrated, for example, in the article by González et al (2021). The authors demonstrate an advanced teleoperation and control system for industrial robots in order to assist the human operator to perform tasks such as sanding, deburring, finishing, grinding, polishing, and so on.…”