Industrial Collaborative Robot Digital Twin integration and Control Using Robot Operating System

Abstract

Standardized and universal solutions for industrial robot integration are increasingly urgent requirements for companies looking for machine interconnectivity, and flexibility in creating tailor made manufacturing systems. These solutions must be supported by modular and open-source components able to easily integrate new control methods and advanced Extended Reality (XR) interfaces. Robot Operating System (ROS) has proven to be a reliable standard for industrial robot integration. ROS compatibility software is provided by many producers and allows for the implementation of modular control units by unifying development practices along the same libraries and methods. Digital Twins (DT) of industrial equipment and processes offer a solid base to develop innovative digital tools relying on synchronization between physical and digital entities and to easily setup intuitive XR interfaces for teleoperation and programming. This work presents the integration of the OMRON TM5-9000 collaborative industrial robot into the IVAR laboratory DT system at Tallinn University of Technology. By using Unity3D game engine and developing a ROS package for the specific machine, the digital model of the collaborative robot is integrated into the existing twin. Synchronization with the real counterpart is provided by MQTT protocol while a robot user interface is developed in Unity and provides robot joints visualization and remote control.

Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions

“…Owing to its high success rates in industrial robot integration, the authors in [168] have established a proof-of-concept AR-integrated system for surgical interventions based on ROS and Unity3D. The work was based on the lightweight KUKA robot, which can be manipulated using the TMFlow software in parallel with pre-defined programs.…”

Video Processing from a Virtual Unmanned Aerial Vehicle: Comparing Two Approaches to Using OpenCV in Unity

“…Simulation through extended reality is gaining more and more attention in manifold domains requiring computer vision like robotic manufacturing [16][17][18], the automotive industry [19,20], video games [21], fitting rooms [22], mulsemedia setups [23], and IoTready WebApps [24], among others. In fact, by creating virtual environments, synthetic training datasets can be generated in a controllable and customizable manner for further solving common computer vision tasks and avoiding the challenges of collecting and annotating data in the real world [25,26].…”