Architecture for Safe Human-Robot Collaboration: Multi-Modal Communication in Virtual Reality for Efficient Task Execution

Abstract

Task-sharing and Human-Robot Collaboration has gained increased attention with the widespread commissioning and usage of collaborative robots. However, recent studies show that the fenceless collaborative robots are not as harmless as they look like. In order to study Human-Robot Interaction scenarios, in a safe manner, we propose to execute the scenario in Virtual Reality simulation and afterwards implement it in real robotic applications (supervised from VR). In addition, this simulated world allows ad-hoc modifications and easy prototyping of different multi-modal communication forms. In this paper we propose an architecture for safe human-robot collaboration and describe a use-case: task of nut screwing, which is executed by the human and the robot together. The nut is hold by the human and the screw is screwed into the nut by the robot (as this part is the repetitive part of the task). The task can be executed in the VR simulation with different input and feedback channels (multi-modal) in order to identify the most efficient communication way between the human and the robot. The different input and output channels are presented in detail.

Extending the Cognitive-Affective Theory of Learning with Media in Virtual Reality Learning: A Structural Equation Modeling Approach

“…VR can afford learners a sense of embodiment and presence that contribute to higher motivation. The motivational path, which proceeds from VR format to motivation, confirms the value of using multi-processing channels on enhancing learners' user experience in a VR environment (Anazco, 2020;Birt et al, 2015;Deng, 2017;Fernandes et al, 2016;Shu et al, 2019;Ta, 2018). Some researchers claimed that users' increased motivation in a new media environment was due to the novelty and this increasing was unsustainable (Clark, 1983;Koch et al, 2018).…”

“…CATLM has been credited in the design of computer simulations, computer-based instruction, online learning, and serious games (e.g., Domagk et al, 2010; Krämer & Bente, 2010; Plass & Schwartz, 2014; Ritterfeld et al, 2009; Sosa et al, 2011). With regards to VR learning, CATLM has been used to define the parameters and evaluation of user experience and user-centered instructional design, such as visualizations (Birt et al, 2015), human-robot collaboration (Shu et al, 2019), teaching concepts, and procedures (Ta, 2018); and to inform pedagogy for increasing learning performance, such as knowledge construction (Spek et al, 2008), knowledge transfer (Petersen et al, 2020), and behavior involvement (Sajjadi et al, 2018).…”

A survey of technologies supporting design of a multimodal interactive robot for military communication

“…Such algorithms may have application in low critical circumstances; however, even the most mundane military tasks cannot be susceptible to algorithmic adjudication. Shu et al (2019) proposed an architecture for safe human robot collaboration on a coordinated and repetitive physical task. The HRI was initiated within a Virtual Reality (VR) simulation and followed by implementation of the tasks with real world robotic applications under VR supervision.…”

“…Shu et al . (2019) proposed an architecture for safe human robot collaboration on a coordinated and repetitive physical task.…”

Human-Robot Collaboration: Safety by Design

“…Most widespread programming of industrial robots can be done by two ways: offline programming or teaching. Both programming methodologies suffer from drawbacks in applications, where human should assist in the robot programming [1].…”