Virtual Reality for Safe Testing and Development in Collaborative Robotics: Challenges and Perspectives

Abstract

Collaborative robots (cobots) could help humans in tasks that are mundane, dangerous or where direct human contact carries risk. Yet, the collaboration between humans and robots is severely limited by the aspects of the safety and comfort of human operators. In this paper, we outline the use of extended reality (XR) as a way to test and develop collaboration with robots. We focus on virtual reality (VR) in simulating collaboration scenarios and the use of cobot digital twins. This is specifically useful in situations that are difficult or even impossible to safely test in real life, such as dangerous scenarios. We describe using XR simulations as a means to evaluate collaboration with robots without putting humans at harm. We show how an XR setting enables combining human behavioral data, subjective self-reports, and biosignals signifying human comfort, stress and cognitive load during collaboration. Several works demonstrate XR can be used to train human operators and provide them with augmented reality (AR) interfaces to enhance their performance with robots. We also provide a first attempt at what could become the basis for a human–robot collaboration testing framework, specifically for designing and testing factors affecting human–robot collaboration. The use of XR has the potential to change the way we design and test cobots, and train cobot operators, in a range of applications: from industry, through healthcare, to space operations.

When Robots Fail—A VR Investigation on Caregivers’ Tolerance towards Communication and Processing Failures

“…VR offers an alternative to provide visual cues that are similar to the real world and creates realistic and immersive environments. Badia and colleagues stated that VR systems that elicit a realistic feeling and appear to be plausible can even create the same behavioral and psychophysiological responses as a real-world interaction [29]. VR has several advantages, but also raises new challenges [30].…”

“…VR has several advantages, but also raises new challenges [30]. Human safety, for example, is crucial when interacting with robots [29]. VR can be used to explore new forms of interactions, as it provides a safe tool for testing HRI without jeopardizing the safety of humans.…”

“…Furthermore, VR allows the testing of multiple virtual robots with different designs in various environments. This does not have to be limited to existing robot systems, as hypothetical robot appearances and behaviors can be implemented as well [29]. Overall, VR provides a less resource-consuming tool (i.e., time and cost) compared to studies with real robots [28].…”

“…In addition to the environment, the presentation of the robot influences the perception and evaluation of robots and the effects on humans [13]. Badia and colleagues have identified variables that can be manipulated and measured in a VR experiment [29]. They concluded that HRI studies in VR offer the assessment of subjective and objective metrics, thereby providing comparable options as real experiments.…”

Use of mixed reality for the training of operators of mobile elevating work platforms with the aim of increasing the level of health and safety at work and reducing training costs

“…Virtual and Mixed Reality devices can provide realistic immersive experiences and are spreading in various industrial sectors (Greci, 2022), showing their true potential either in the production processes in combination with other technologies of the Industry 4.0 paradigm, such as collaborative robotics and rapid prototyping (Badia, et al, 2022;Schroeter & Gerber, 2018), or in interactive simulation applications for educational, training and remote maintenance purposes (Casini,2022;Mourtzis et al 2020, Xie, et al, 2021.…”