Towards the development of an intuitive teleoperation system for human support robot using a VR device

Nakanishi, Jun; Itadera, Shunki; Aoyama, Tadayoshi; Hasegawa, Yasuhisa

doi:10.1080/01691864.2020.1813623

Cited by 25 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In conventional telemanipulation systems, the user controls the remote robotic system with a joystick, gamepad, keyboard and mouse, or 3D mouse and simultaneously receives visual feedback from 2D displays [25,26]. Robot control is not intuitive and natural to the user [27,28]. The mismatch between the range of user control space and the limits of input device workspace can increase the difficulty of telemanipulation and lead to poor operation [29,30].…”

Section: Intuitive and Natural Teleoperationmentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

show abstract

Section: Intuitive and Natural Teleoperationmentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…where J(q) is the Jacobi matrix of the inspection robots that can obtained by using differential transformation method according to Literature. 27 The Jacobi matrix is shown in equation ( 16) For the eigenvalues of the Cartesian inertia matrix, the quadratic equation is shown in equation ( 17)…”

Section: The Evaluation Indexmentioning

confidence: 99%

“…The expression of the eigenvalues of the Cartesian inertia matrix is shown in equation (15)where J ( q ) is the Jacobi matrix of the inspection robots that can obtained by using differential transformation method according to Literature. 27 The Jacobi matrix is shown in equation (16)For the eigenvalues of the Cartesian inertia matrix, the quadratic equation is shown in equation (17)where x is the feature vectors of the Cartesian inertia matrix.…”

Section: Dynamic Model and Evaluation Indexmentioning

confidence: 99%

The climbing performance analysis of a robot for power line inspection with retractable double serial manipulators

Shang

et al. 2021

Proceedings of the Institution of Mechanical Engineers, Part C:

View full text Add to dashboard Cite

A power line inspection robot has to overcome many kinds of obstacles in inspection processes. The strain clamp is the obstacles difficult for inspection robots to overcome. The inspection robot needs to have a particular climbing ability to overcome the strain clamp. Therefore, the ability to climb power lines is the key point of the inspection robot’s design. An inspection robot with retractable double serial manipulators is proposed to improve the climbing and obstacle-crossing ability. Besides, this paper shows that the inspection robot is more suitable for climbing from static analysis and dynamic evaluation index. Firstly, the obstacle-crossing processes and structures of the inspection robot are introduced. Next, the static analysis is carried out when inspection robot climbs the power lines. The static analysis shows that the new inspection robot has a smaller driving torque. What’s more, the dynamic model of the inspection robot is established by Lagrange’s dynamical equations. By constructing the dynamic evaluation indexes, the inspection robot with retractable arms performs better dynamic characteristics. Finally, a prototype robot is carried out to cross obstacles and climb up power lines. The results show that the inspection robot can overcome different obstacles and has a good climbing performance.

show abstract

“…The mutual communication between each node and its parent node is realized by a two-way information exchange to compensate for the clock deviation. The flooding time synchronization protocol (FTSP) proposed by Nakanishi et al (2020) assigns an ID number to each sensor node, completes the clock synchronization between the sending node and the receiving node through broadcasting synchronization packets, realizes the clock synchronization of the whole network by hierarchical classification, and uses linear regression deviation compensation to compensate for the relevant error source. The time diffusion protocol (TDP) proposed by Outón et al (2019) is based on the iterative weighted average method and uses the information diffused by the sensor nodes involved in the whole network in the synchronization process.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Door Opening Control Algorithm of Bio-Inspired Mobile Manipulator Based on Synchronous Sensing

Wu¹,

Liu²

2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

At present, the research of robot door opening method is basically realized by identifying the door handle through the synchronous sensing system on the premise that the bio-inspired mobile manipulator is located in front of the door. An adaptive door opening strategy of a bio-inspired mobile manipulator based on a synchronous sensing system is proposed. Firstly, the random delay distribution in clock synchronization technology is analyzed in detail, and its distribution is verified on the experimental platform of adjacent nodes. Based on the Gaussian distribution of random delay, the relative frequency offset and relative phase offset of adjacent nodes are calculated. The clock synchronization of network cable sensor nodes is realized. Secondly, based on the information data of synchronous sensing system, this article realizes target detection and tracking based on depth network. In addition, based on the sliding mode control theory, the dynamic model of the nonholonomic bio-inspired mobile manipulator is applied. Finally, a robust adaptive sliding mode control method for nonlinear systems with input gain uncertainty and unmatched uncertainty is proposed by combining adaptive backstepping with sliding mode control. By adding sliding mode control in the last step of adaptive backstepping, the uncertainty of the system is compensated, and the system trajectory is maintained on the specified sliding mode manifold. The tracking control and stability control of the nonholonomic bio-inspired mobile manipulator are simulated. The experimental and simulation results show that the control method proposed in this article is effective and feasible.

show abstract

Towards the development of an intuitive teleoperation system for human support robot using a VR device

Cited by 25 publications

References 21 publications

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

The climbing performance analysis of a robot for power line inspection with retractable double serial manipulators

Adaptive Door Opening Control Algorithm of Bio-Inspired Mobile Manipulator Based on Synchronous Sensing

Contact Info

Product

Resources

About