An Augmented Reality System to Support Fault Visualization in Industrial Robotic Tasks

Abstract

The digitalization is transforming the very nature of factories, from automated systems to intelligent ones. In this process, industrial robots play a key role. Even if repeatability, precision and velocity of the industrial manipulators enable reaching considerable production levels, factories are required to face an increasingly competitive market, which requires being able to dynamically adapt to different situations and conditions. Hence, facilities are moving toward systems that rely on the collaboration between humans and machines. Human workers should understand the behavior of the robots, placing trust in them to properly collaborate. If a fault occurs on a manipulator, its movements are suddenly stopped for security reasons, thus workers may not be able to understand what happened to the robot. Therefore, the operators' stress and anxiety may increase, compromising the human-robot collaborative scenario. This work fits in this context and it proposes an adaptive Augmented Reality system to display industrial robot faults by means of the Microsoft HoloLens device. Starting from the methodology employed to identify which virtual metaphors best evoke robot faults, an adaptive modality is presented to dynamically display the metaphors in positions close to the fault location, always visible from the user and not occluded by the manipulator. A comparison with a non adaptive modality is proposed to assess the effectiveness of the adaptive solution. Results show that the adaptive modality allows users to recognize faults faster and with fewer movements than the non adaptive one, thus overcoming the limitation of the narrow field-of-view of the HoloLens device.

A Review on Augmented Reality Tracking Methods for Maintenance of Robots

“…Avalle et al [21] proposed a fault visualization on a robotic arm using a HMD (Hololens) as part of humanrobot collaboration (HRC) systems related to safety. The tracking method used is marker-based to display the virtual robotic arm.…”

Augmented Reality for Robotics: A Review

“…An AR interface designed to provide assistance during robotic welding was presented by Ong et al [87]. Another AR programming framework for welding robots was developed by Avalle et al [88]. This tool helped a new user to complete complex welding tasks without having expertise in robotics.…”

3d Surveying, Semantic Enrichment and Virtual Access of Large Cultural Heritage

“…This blend combines capabilities from the worlds of VR (in which the user is fully immersed in the computer graphic world) and standard AR, resulting in an environment where digital and physical coexists and interact (Teruggi and Fassi, 2021). MR and AR have been used in a variety of fields: assembly and disassembly (De Pace et al, 2018), human-robot interaction (Avalle et al, 2019), navigation (Fraga-Lamas et al, 2018), medicine (Desselle et al, 2020;Brun et al, 2019), Architecture Engineering Construction and Operations -AECO (Bae et al, 2015;Golparvard-Fard and Ham, 2014;Alam et al, 2017) as well as cultural heritage (De Carolis et al, 2018;Carrozzino et al, 2019).…”