Interactive Robots Control Using Mixed Reality

Ostanin, Mikhail; Yagfarov, Rauf; Klimchik, Alexandr

doi:10.1016/j.ifacol.2019.11.307

Cited by 26 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In MR, the hierarchy of reality spans from partial sensory input to a proximity infinitely close to immersive reality [97]. The reviewed MR-enhanced intuitive and immersive teleoperation schemes in this article fall into the category of partially superimposing realworld visual inputs onto computer-generated virtual content.…”

Section: Reality-virtuality Continuummentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

show abstract

Section: Reality-virtuality Continuummentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Following the introduction of augmented reality (AR), image overlay of 3D holograms onto the users vision has been made possible. AR systems have seen applications in a wide range of working scenarios, fulfilling the purposes of motion planning [12], control [13], and visualisation [14]. The main benefits of having an AR assistive system was well summarised by Makhataeva et al's review [15].…”

Section: The Significance Of Extended Reality In Hrimentioning

confidence: 99%

Augmented Reality-Assisted Reconfiguration and Workspace Visualization of Malleable Robots: Workspace Modification Through Holographic Guidance

Ranne

Clark

Rojas

2022

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

Malleable robots are a type of reconfigurable serial robot capable of adapting their topology, through the use of variable stiffness malleable links, to desired tasks and workspaces by varying the relative positioning between their revolute joints. However, their reconfiguration is non-trivial, lacking intuitive communication between the human and the robot, and a method of efficiently aligning the end-effector to a desired position. In this paper, we present the design of an interactive augmented reality alignment interface, which helps a malleable robot understand the users task requirements, visualises to the user the requested robot configuration and its workspace, and guides the user in reconfiguring the robot to achieve that configuration. Through motion tracking of a physical two-degree-of-freedom malleable robot, which can achieve an infinite number of workspaces, we compute the accuracy of the system in terms of initial calibration and overall accuracy, and demonstrate its viability. Results demonstrated a good performance with an average repositioning accuracy of 9.64±2.06 mm, and an average base alignment accuracy of 10.54±4.32 mm, in an environment of size 2000mm x 2000mm x 1200mm.

show abstract

“…Figure: [263] -Household Task authoring home automation [53,111,135,163,177,191,212,228,263,327,365,387,480], item movement and delivery [76,108,209,265], displaying cartoon-art while sweeping [481], multi-purpose table [430] Photography drone photography [114,171], Advertisement mid-air advertisement [317,382,418,475] Wearable Interactive Devices haptic interaction [443], fog screens [418], head-worn projector for sharable AR scenes [168] Assistance and companionship elder care [65], personal assistant [239,367,368] Tour and exhibition guide tour guide [58,295], museum exhibition guide [168,368], guiding crowds [475], indoor building guide [89], museum interactive display [112] Industry 166 Figure: [21] -Manufacturing and Assembly joint assembly and manufacturing [21, 30, 43, 44, 80, 138-140, 161, 194, 231, 274, 283, 289, 297, 327, 366, 376, 396, 397, 410, 435, 454, 456, 493], grasping and manipulation [24-27, ...…”

Section: Domestic and Everyday Use 35mentioning

confidence: 99%

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces

Suzuki,

Karim,

Xia

et al. 2022

Preprint

View full text Add to dashboard Cite

Interactive Robots Control Using Mixed Reality

Cited by 26 publications

References 10 publications

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Augmented Reality-Assisted Reconfiguration and Workspace Visualization of Malleable Robots: Workspace Modification Through Holographic Guidance

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces

Contact Info

Product

Resources

About