The Vicarios Virtual Reality Interface for Remote Robotic Teleoperation

Abstract

Intuitive interaction is the cornerstone of accurate and effective performance in remote robotic teleoperation. It requires high-fidelity in control actions as well as perception (vision, haptic, and other sensory feedback) of the remote environment. This paper presents Vicarios, a Virtual Reality (VR) based interface with the aim of facilitating intuitive real-time remote teleoperation, while utilizing the inherent benefits of VR, including immersive visualization, freedom of user viewpoint selection, and fluidity of interaction through natural action interfaces. Vicarios aims to enhance the situational awareness, using the concept of viewpoint-independent mapping between the operator and the remote scene, thereby giving the operator better control in the perception-action loop. The article describes the overall system of Vicarios, with its software, hardware, and communication framework. A comparative user study quantifies the impact of the interface and its features, including immersion and instantaneous user viewpoint changes, termed “teleporting”, on users’ performance. The results show that users’ performance with the VR-based interface was either similar to or better than the baseline condition of traditional stereo video feedback, approving the realistic nature of the Vicarios interface. Furthermore, including the teleporting feature in VR significantly improved participants’ performance and their appreciation for it, which was evident in the post-questionnaire results. Vicarios capitalizes on the intuitiveness and flexibility of VR to improve accuracy in remote teleoperation.

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

“…Several studies have explored the integration of VR in teleoperation systems. For example, Abdeldjallil Naceri et al [31] proposed a real-time teleoperation platform that uses an HTC Vive Pro headset, UDP sockets, and the ROS Bridge protocol to communicate between the VR headset and the robot. They also integrated an RGB-D camera to provide 3D pose and dimension information of the remote workstation and object.…”

DARLOS: a lightweight dual-arm robotic live-line operation system for autonomous high-voltage distribution grid maintenance

“…On the other hand, the level of artificial intelligence embedded in such systems was also limited, resulting in most early systems being controlled remotely by human operators on the ground via teleoperation (Penin et al , 1998; Udod et al , 1998; Soler and Guillet, 1993). Since 2000, a great deal of research has been carried out in target recognition, positioning and attitude estimation (Feng and Zhang, 2021; Yan et al , 2020; Zhao et al , 2022; Wu et al , 2021; Naceri et al , 2021). The performance of live-line operation robot system has been greatly improved, even as the operation mode develops from pure teleoperation to automatic control (Zhu et al , 2019; Tani et al , 2015; Kato et al , 2016; Ito et al , 2010; Chen et al , 2023).…”

Intelligent power distribution live‐line operation robot systems based on stereo camera

“…The position of the operator has also changed from high-altitude close-range operation to operation on the ground. This condition guarantees the safety of operators to a greater extent but puts forward higher requirements on technologies, such as real-time visual feedback, target recognition, and positioning [18][19][20].…”