Intuitive and Safe Interaction in Multi-User Human Robot Collaboration Environments through Augmented Reality Displays

Abstract

As autonomous collaborative robots are more widely used in work environments alongside humans it is of great importance to facilitate the communication between people and robotic systems, in a way that promotes safety and productivity. To this end, we propose an Augmented Reality (AR) based system that allows workers in a human-robot collaborative environment to interact with a robot while also receiving information regarding the robot state and plans that relate to the human's safety and trust, such as the intended movement of the robotic arm or the navigation plan of the mobile platform. To evaluate the effectiveness of the proposed system we conducted experiments with 13 participants, where two users had to work in the same workspace while being assisted by a mobile manipulator. We measured the task completion time as well as the robot idle time using our ARbased human-robot interaction system and compared them to a conventional setup without the use of augmented reality. Additional, subjective evaluations related to user satisfaction, system usability, perceived safety and trust showed that users assessed the system in a positive way and preferred AR visualization over more traditional interfaces.

Augmented Reality for Human–Robot Collaboration and Cooperation in Industrial Applications: A Systematic Literature Review

“…Despite that, according to Fernández del Amo et al [ 78 ], the quality assessment is a process prone to bias itself; therefore, the mean and standard deviation for each criterion is provided as a tool for numerical validation of a potential author bias in the assessment process. Not considering the quantitative criteria (QC1 and QC2), the papers that were perceived with the subjectively higher quality were Kalpagam et al [ 49 ] and Chan et al [ 88 ], followed by Materna et al [ 89 ], Hietanen et al [ 51 ], and Tsamis et al [ 90 ], with a score of 7.5 out of 8 for the first two, and 7 out of 8 for the following three, respectively. If one considers the quantitative criteria, the order of relevance changes to Hietanen et al [ 51 ], Kalpagam et al [ 49 ], Chan et al [ 88 ], Materna et al [ 89 ], and Michalos et al [ 91 ] figuring out in the last place instead of Hietanen et al [ 51 ] on the top 5, for a small score difference of 0.025 points.…”

“…Andersen et al [ 93 ] used a UR cobot for a pick and place task, where the robot and the operator, alternately, change the position of a box over a table. On the other hand, Tsamis et al [ 90 ] also uses a UR manipulator for a pick-and-place task, but for exchanging parts from a workstation to a different one. Liu and Wang [ 94 ], Argyrou et al [ 96 ], Hietanen et al [ 51 ] uses a cobot to aid the operator on a car engine assembly process, performing parts handling and mounting.…”

“…Tsamis et al [ 90 ] also has a robotic manipulator mounted over a mobile platform. The developed augmented reality application enables the user to visualize the mobile platform and the manipulator’s planned trajectories by interpolating holographic 3D green spheres accordingly to the calculated waypoints.…”

“…Therefore, they can be classified as passive approaches. On the other hand, as discussed above, Vogel et al [ 109 ], Vogel et al [ 95 ], Argyrou et al [ 96 ], Ji et al [ 101 ], Bolano et al [ 106 ], Lotsaris et al [ 111 ] and Tsamis et al [ 90 ] presented active approaches towards assuring the operator’s safety.…”

“…Moreover, it also provides the user notifications about the activity’s current status. Similarly, in Tsamis et al [ 90 ], the robot’s current status is given as feedback to the user informing whether the motion execution or planning has failed or succeeded.…”