Research on Teleoperated Virtual Reality Human–Robot Five-Dimensional Collaboration System

Zhang, Qinglei; Liu, Qinghao; Duan, Jianguo; Qin, Jiyun

doi:10.3390/biomimetics8080605

Cited by 5 publications

(2 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human action recognition technology has been widely used in the fields of bionic robots, medical care, video surveillance, intelligent transportation, and human–robot interaction [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ]. Skeletons are made up of a variety of different shapes, with complex internal and external structures, which play a role in supporting and protecting the human body.…”

Section: Introductionmentioning

confidence: 99%

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Hu,

Liu

2024

Biomimetics

View full text Add to dashboard Cite

Skeleton-based human interaction recognition is a challenging task in the field of vision and image processing. Graph Convolutional Networks (GCNs) achieved remarkable performance by modeling the human skeleton as a topology. However, existing GCN-based methods have two problems: (1) Existing frameworks cannot effectively take advantage of the complementary features of different skeletal modalities. There is no information transfer channel between various specific modalities. (2) Limited by the structure of the skeleton topology, it is hard to capture and learn the information about two-person interactions. To solve these problems, inspired by the human visual neural network, we propose a multi-modal enhancement transformer (ME-Former) network for skeleton-based human interaction recognition. ME-Former includes a multi-modal enhancement module (ME) and a context progressive fusion block (CPF). More specifically, each ME module consists of a multi-head cross-modal attention block (MH-CA) and a two-person hypergraph self-attention block (TH-SA), which are responsible for enhancing the skeleton features of a specific modality from other skeletal modalities and modeling spatial dependencies between joints using the specific modality, respectively. In addition, we propose a two-person skeleton topology and a two-person hypergraph representation. The TH-SA block can embed their structural information into the self-attention to better learn two-person interaction. The CPF block is capable of progressively transforming the features of different skeletal modalities from low-level features to higher-order global contexts, making the enhancement process more efficient. Extensive experiments on benchmark NTU-RGB+D 60 and NTU-RGB+D 120 datasets consistently verify the effectiveness of our proposed ME-Former by outperforming state-of-the-art methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Hu,

Liu

2024

Biomimetics

View full text Add to dashboard Cite

show abstract

“…This contributes to improved task performance and overall efficiency in various teleoperation applications. The applications of teleoperation systems are versatile, such as in unmanned aerial vehicles [3][4][5], virtual reality [6,7], medical training [8][9][10], and telesurgery [11][12][13][14][15][16].…”

Section: Introductionmentioning

confidence: 99%

Robust Adaptive-Sliding-Mode Control for Teleoperation Systems with Time-Varying Delays and Uncertainties

Chang,

Yang,

Lin

2024

Robotics

View full text Add to dashboard Cite

Master–slave teleoperation systems with haptic feedback enable human operators to interact with objects or perform tasks in remote environments. This paper presents a sliding-mode control scheme tailored for bilateral teleoperation systems operating in the presence of unknown uncertainties and time-varying delays. To address unknown but bounded uncertainties, adaptive laws are derived alongside controller design. Additionally, a linear matrix inequality is solved to determine the allowable bound of delays. Stability of the closed-loop system is ensured through Lyapunov–Krasovskii functional analysis. Two-degree-of-freedom mechanisms are self-built as haptic devices. Free-motion and force-perception scenarios are examined, with experimental results validating and comparing performances. The proposed adaptive-sliding-control method increases the position performance from 58.48% to 82.55% and the force performance from 83.48% to 99.77%. The proposed control scheme demonstrates enhanced position tracking and force perception in bilateral teleoperation systems.

show abstract

Analysis of MR–VR tele-operation methods for legged-manipulator robots

Cruz Ulloa,

Domínguez,

del Cerro

et al. 2024

Virtual Reality

View full text Add to dashboard Cite

The development of immersive technologies in recent years has facilitated the control and execution of tasks at a high level of complexity in robotic systems. On the other hand, exploration and manipulation tasks in unknown environments have been one of the main challenges in search and rescue (SAR) robotics. Due to the complexity and uncertainty involved in autonomous manipulation tasks in unstructured environments, these are usually tele-operated initially. This article addresses a comparative study between Mixed Reality (MR—Hololens) and Virtual Reality (VR—HTC-Vive) methods for teleoperating legged-manipulator robots in the context of search and rescue. For this purpose, a teleoperation robotics method was established to address the comparison, developing VR–MR interfaces with the same contextualization and operational functionality for mission management and robot control of a robotic set composed of a quadrupedal robot equipped with a 6 degrees of freedom (6DoF) manipulator, by a user using hand gestures. A set of metrics is proposed for the comparative evaluation of the interfaces considering parameters that allow analyzing operability in the context of the mission (latencies, physical parameters of the equipment, etc.), as well as from the aspect of operator performance (required training, confidence levels, etc.). The experimental phase was conducted using both on-site and remote operations to evaluate and categorize the advantages and disadvantages of each method.

show abstract

Research on Teleoperated Virtual Reality Human–Robot Five-Dimensional Collaboration System

Cited by 5 publications

References 42 publications

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Multi-Modal Enhancement Transformer Network for Skeleton-Based Human Interaction Recognition

Robust Adaptive-Sliding-Mode Control for Teleoperation Systems with Time-Varying Delays and Uncertainties

Analysis of MR–VR tele-operation methods for legged-manipulator robots

Contact Info

Product

Resources

About