Implementing and Assessing a Remote Teleoperation Setup with a Digital Twin Using Cloud Networking

Pulgarin, Erwin Jose Lopez; Niu, Hongmei; Herrmann, Guido; Carrasco, Joaquín

doi:10.1007/978-3-031-15908-4_19

Cited by 3 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach allows the user to send data to the server using protocols such as HTTP, TCP, and UDP. Another serverless approach could also be implemented, such as Erwin Jose Lopez Pulgarin et al [59] digital twin platform where an Amazon AWS EC2 virtual machine is deployed for IPsec VPN, allowing remote connection with similar performance as in a wired setup. MQTT protocol is another approach with serverless architecture.…”

Section: Literature Reviewmentioning

confidence: 99%

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

Luu,

Nguyen,

Tran

et al. 2024

Preprint

View full text Add to dashboard Cite

During the early stages of the Industry 4.0 era, teleoperation with human involvement in manufacturing processes and robot operation remained important. The existing teleoperation technologies have limitations in providing operator with spatial awareness and initiative control mechanism, making their applications difficult in tasks with high complexity and precision. This study implements the advancements from the Internet of Things (IoT) and Virtual Reality (VR) technology to improve the real-time teleoperation for industrial robots. The proposed design utilizes the benefits of the Message Queuing Telemetry Transport (MQTT) protocol in conjunction with a cloud broker, which has not been addressed significantly in previous studies for teleoperation. This setup enables the system to efficiently handle substantial amounts of data in short periods of time, allowing for an uninterrupted user experience while maintaining a consistent connection regardless of distance. Real-time robot control is accomplished by tracking the user's hand movements with a VR controller while providing the user with visual feedback from the VR environment. The incorporation of VR technology intends to improve the intuitiveness of controlling robot operations, provide more responsive feedback, and improve the quality of work performed by operators. Practical experiments are conducted with an industrial robot to observe the performance against the system under different internet conditions and different QoS setups. The evaluation criteria primarily concern latency and accuracy. The experiment aims to evaluate the effectiveness and scalability of the suggested system and to offer significant insight into its advantages and limitations relative to existing control system through systematic investigation.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

Luu,

Nguyen,

Tran

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, the DDS protocol is used to publish sensor data thanks to the gateway from the robot and subscribe to commands from the maintainer, allowing real-time, bi-directional communication between the remote robot and the maintainer. DT technology can be used in conjunction with telexistence to provide a more realistic and immersive experience for the maintainer controlling the remote robot [14]. A DT is a virtual representation of a physical object or system, which can be used to simulate and analyze its behavior and performance [15].…”

Section: Telexistence Maintenance Framework Strategiesmentioning

confidence: 99%

Telexistence-Based Remote Maintenance for Marine Engineers

Mazeas

Erkoyuncu

Noël

2023

2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

View full text Add to dashboard Cite

Remote work practice has seen significant developments in the field of maintenance, which is beneficial in the maritime sector. The defense industry is investigating the utilization of unmanned vehicle systems (UVS) to fulfil the demands of efficiency, safety, and field tasks that heavily rely on equipment usage. Telexistence technology affords marine engineers the capability to conduct inspections and repairs as if physically present. This study presents a scalable framework for evaluating the feasibility of conducting traditional machinery space maintenance through the implementation of telexistence capabilities. Our case study employs a physical robot for real-motion behaviors and a simulated virtual environment. Maintenance scenarios were developed through responses to a contextual questionnaire by two experts. In future steps, the study will compare telexistence with direct teleoperation in terms of presence, workload, and usability for vessel machinery space maintenance.

show abstract

“…Ensuring data accuracy and calibration poses another significant challenge, as digital twins heavily depend on precise sensor data to accurately represent the state of the physical robot [130]. Additionally, creating detailed and accurate digital twin models for complex robotic systems presents a formidable obstacle [131,132]. The complexity of the physical system must be faithfully mirrored in the digital twin to provide valuable and meaningful insights, adding to the complexity of the integration process [133].…”

Section: Mixed Reality-based Vision Mapping and Mergingmentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

show abstract

Implementing and Assessing a Remote Teleoperation Setup with a Digital Twin Using Cloud Networking

Cited by 3 publications

References 15 publications

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

Telexistence-Based Remote Maintenance for Marine Engineers

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Contact Info

Product

Resources

About