Review on existing VR/AR solutions in human–robot collaboration

Dianatfar, Morteza; Latokartano, Jyrki; Lanz, Minna

doi:10.1016/j.procir.2020.05.259

Cited by 101 publications

(40 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In recent decades, AR technology has been frequently used for human-robot collaboration [33]. Previous work has mainly focused on the use of HMDs, MAR, SAR and the visualization of the robot motion intent [9,34,35].…”

Section: Augmented Reality In Human-robot Collaborationmentioning

confidence: 99%

My Caregiver the Cobot: Comparing Visualization Techniques to Effectively Communicate Cobot Perception to People with Physical Impairments

Pascher

Kirill

Til

et al. 2022

Sensors

View full text Add to dashboard Cite

Nowadays, robots are found in a growing number of areas where they collaborate closely with humans. Enabled by lightweight materials and safety sensors, these cobots are gaining increasing popularity in domestic care, where they support people with physical impairments in their everyday lives. However, when cobots perform actions autonomously, it remains challenging for human collaborators to understand and predict their behavior, which is crucial for achieving trust and user acceptance. One significant aspect of predicting cobot behavior is understanding their perception and comprehending how they “see” the world. To tackle this challenge, we compared three different visualization techniques for Spatial Augmented Reality. All of these communicate cobot perception by visually indicating which objects in the cobot’s surrounding have been identified by their sensors. We compared the well-established visualizations Wedge and Halo against our proposed visualization Line in a remote user experiment with participants suffering from physical impairments. In a second remote experiment, we validated these findings with a broader non-specific user base. Our findings show that Line, a lower complexity visualization, results in significantly faster reaction times compared to Halo, and lower task load compared to both Wedge and Halo. Overall, users prefer Line as a more straightforward visualization. In Spatial Augmented Reality, with its known disadvantage of limited projection area size, established off-screen visualizations are not effective in communicating cobot perception and Line presents an easy-to-understand alternative.

show abstract

Section: Augmented Reality In Human-robot Collaborationmentioning

confidence: 99%

My Caregiver the Cobot: Comparing Visualization Techniques to Effectively Communicate Cobot Perception to People with Physical Impairments

Pascher

Kirill

Til

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Specifically, the lack of scalable AR applications in real production lines is pinpointed [27] alongside the necessity for quality and reliability improvements of the proposed AR solutions [9,28]. Hardware limitations are also present regarding the AR devices' performance after extensive use and the battery consumption during high processing power tasks, such as image recognition [4,25]. Furthermore, the important role of the human factor in the process of designing and developing AR applications is highlighted.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The existence of multiple markers is an alternative for tackling the issue, but it increases the system's complexity and often leads to unsatisfactory results. A 2D camera for image recognition could be used for increasing accuracy but, unfortunately, dust and poor lighting often hinder the process [4]. An interesting approach is to use spatial registration to localize users and apply this output for feeding the AR application [5].…”

Section: Introductionmentioning

confidence: 99%

An Augmented Reality Symbiosis Software Tool for Sustainable Logistics Activities

2021

View full text Add to dashboard Cite

Augmented Reality (AR) is an emerging technology in the Industry 4.0 and Logistics 4.0 contexts with an important role in man–machine symbiosis scenarios. Practitioners, although already acquainted with AR technology, are reluctant to adopt AR applications in industrial operations. This stems from the fact that a direct connection that is important for the management of sustainability goals is missing. Moreover, such a connection with economic, social, and environmental sustainability parameters sparsely appears in the AR literature. The proposed research, on one hand, presents an innovative architecture for a stable and scalable AR application that extents state-of-the-art solutions and, on the other hand, attempts to study AR technology within the framework of a sustainable business strategy. The developed system utilizes the Robot Operating System (ROS) alongside an AR mobile application to present an employee navigation scenario in warehouses and production lines. ROS is responsible for mapping the industrial facility, while the AR mobile application identifies the surrounding environment, along with a Real-Time Location System localizes employees in the facility. Finally, ROS identifies the shortest path between the employee and the destination point, while the AR mobile application presents the virtual path for reaching the destination.

show abstract

“…This is perhaps best exemplified by the variety of modalities available for interaction and the magnitude of differences in industrial environments, tasks and contexts. Several different modalities have been utilized for communication, as demonstrated for gestures ( Liu and Wang, 2018 ), augmented and virtual reality ( Dianatfar et al, 2021 ), verbal and non-verbal communication ( Mavridis, 2015 ) and physical interaction ( Ogenyi et al, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

Coordinating Shared Tasks in Human-Robot Collaboration by Commands

et al. 2021

View full text Add to dashboard Cite

Human-robot collaboration is gaining more and more interest in industrial settings, as collaborative robots are considered safe and robot actions can be programmed easily by, for example, physical interaction. Despite this, robot programming mostly focuses on automated robot motions and interactive tasks or coordination between human and robot still requires additional developments. For example, the selection of which tasks or actions a robot should do next might not be known beforehand or might change at the last moment. Within a human-robot collaborative setting, the coordination of complex shared tasks, is therefore more suited to a human, where a robot would act upon requested commands.In this work we explore the utilization of commands to coordinate a shared task between a human and a robot, in a shared work space. Based on a known set of higher-level actions (e.g., pick-and-placement, hand-over, kitting) and the commands that trigger them, both a speech-based and graphical command-based interface are developed to investigate its use. While speech-based interaction might be more intuitive for coordination, in industrial settings background sounds and noise might hinder its capabilities. The graphical command-based interface circumvents this, while still demonstrating the capabilities of coordination. The developed architecture follows a knowledge-based approach, where the actions available to the robot are checked at runtime whether they suit the task and the current state of the world. Experimental results on industrially relevant assembly, kitting and hand-over tasks in a laboratory setting demonstrate that graphical command-based and speech-based coordination with high-level commands is effective for collaboration between a human and a robot.

show abstract

Review on existing VR/AR solutions in human–robot collaboration

Cited by 101 publications

References 14 publications

My Caregiver the Cobot: Comparing Visualization Techniques to Effectively Communicate Cobot Perception to People with Physical Impairments

My Caregiver the Cobot: Comparing Visualization Techniques to Effectively Communicate Cobot Perception to People with Physical Impairments

An Augmented Reality Symbiosis Software Tool for Sustainable Logistics Activities

Coordinating Shared Tasks in Human-Robot Collaboration by Commands

Contact Info

Product

Resources

About