Towards a Virtual Reality Interface for Remote Robotic Teleoperation

Naceri, Abdeldjallil; Mazzanti, Dario; Bimbo, João; Prattichizzo, Domenico; Caldwell, Darwin G.; Mattos, Leonardo S.; Deshpande, Nikhil

doi:10.1109/icar46387.2019.8981649

Cited by 30 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several recent projects have focused on improving human-robot collaboration using virtual, augmented and mixed reality (VAM) devices. Teleoperation (remotely controlling the robot) is one of the most common forms of human-robot interaction [17,18,22]. Ostatin [20] and Togias [24] both use VR to plan the robotic arm's trajectory.…”

Section: Related Workmentioning

confidence: 99%

What you see is (not) what you get: A VR Framework for Correcting Robot Errors

Wozniak,

Stower,

Jensfelt

et al. 2023

Preprint

View full text Add to dashboard Cite

Many solutions tailored for intuitive visualization or teleoperation of virtual, augmented and mixed (VAM) reality systems are not robust to robot failures, such as the inability to detect and recognize objects in the environment or planning unsafe trajectories. In this paper, we present a novel virtual reality (VR) framework where users can (i) recognize when the robot has failed to detect a realworld object, (ii) correct the error in VR, (iii) modify proposed object trajectories and, (iv) implement behaviors on a real-world robot. Finally, we propose a user study aimed at testing the efficacy of our framework. Project materials can be found in the OSF repository 1 . CCS CONCEPTS• Human-centered computing → Virtual reality.

show abstract

Section: Related Workmentioning

confidence: 99%

What you see is (not) what you get: A VR Framework for Correcting Robot Errors

Wozniak,

Stower,

Jensfelt

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…• We employed a VR-based user interface that provides a better perception of the PT's surroundings and real-time user interaction with the DTs. • We presented and evaluated a robot navigation technique that uses DTs and a VR-based user interface, Our method is not constrained by the limitations of camera-based navigation techniques such as restricted field-of-view, low-quality video feedback, communication delay, twodimensional (2D) prospect, poor performance in low lighting, visual data transmission, and processing load [43]- [46]. • We developed a real-time mechanism to detect and visualize stationary and moving obstacles and their distances from the PT, allowing situational awareness of the RE.…”

Section: B Contributionmentioning

confidence: 99%

Digital-Twins-Based Internet of Robotic Things for Remote Health Monitoring of COVID-19 Patients

Khan

Ullah

Khan

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

The deadly coronavirus disease (COVID-19) has highlighted the importance of remote health monitoring (RHM). The digital twins (DTs) paradigm enables RHM by creating a virtual replica that receives data from the physical asset, representing its real-world behavior. However, DTs use passive internet of things (IoT) sensors, which limit their potential to a specific location or entity. This problem can be addressed by using the internet of robotic things (IoRT), which combines robotics and IoT, allowing the robotic things (RTs) to navigate in a particular environment and connect to IoT devices in the vicinity. Implementing DTs in IoRT, creates a virtual replica (virtual twin) that receives real-time data from the physical RT (physical twin) to mirror its status. However, DTs require a user interface for real-time interaction and visualization. Virtual reality (VR) can be used as an interface due to its natural ability to visualize and interact with DTs. This research proposes a real-time system for RHM of COVID-19 patients using the DTs-based IoRT and VR-based user interface. It also presents and evaluates robot navigation performance, which is vital for remote monitoring. The virtual twin (VT) operates the physical twin (PT) in the real environment (RE), which collects data from the patient-mounted sensors and transmits it to the control service to visualize in VR for medical examination. The system prevents direct interaction of medical staff with contaminated patients, protecting them from infection and stress. The experimental results verify the monitoring data quality (accuracy, completeness, timeliness) and high accuracy of PT's navigation.

show abstract

“…While in this use case, the robot operator is present alongside the robot, VAM techniques can also be used to help users to remotely control robot manipulation. Naceri et al [59], for example, present a VR interface for real-time robot teleoperation. In this interface, the remote environment is visualized using streaming External Sensor Images and Videos and External Sensor 3D Data, to make robot teleoperation more effective.…”

Section: Robot Manipulationmentioning

confidence: 99%

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction: A Survey and Virtual Design Element Taxonomy

Walker¹,

Phung²,

Chakraborti³

et al. 2022

Preprint

View full text Add to dashboard Cite

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in HRI research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this paper, we present a novel taxonomic framework for different types of VAM-HRI interfaces, composed of four main categories of virtual design elements (VDEs). We present and justify our taxonomy and explain how its elements have been developed over the last 30 years as well as the current directions VAM-HRI is headed in the coming decade. CCS Concepts: • Human-centered computing → Virtual reality; Mixed / augmented reality; User interface design; • Computer systems organization → External interfaces for robotics.

show abstract

Towards a Virtual Reality Interface for Remote Robotic Teleoperation

Cited by 30 publications

References 10 publications

What you see is (not) what you get: A VR Framework for Correcting Robot Errors

What you see is (not) what you get: A VR Framework for Correcting Robot Errors

Digital-Twins-Based Internet of Robotic Things for Remote Health Monitoring of COVID-19 Patients

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction: A Survey and Virtual Design Element Taxonomy

Contact Info

Product

Resources

About