End-User Robot Programming Using Mixed Reality

Yitzhak, Gadre, Samir; Rosen, Eric; Chien, Gary W.; Phillips, Elizabeth; Tellex, Stefanie; Konidaris, George

doi:10.1109/icra.2019.8793988

Cited by 56 publications

(28 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…-The proposed AR-based interface is equipped with full capabilities to be used as an alternative to TP interfaces for teleoperation tasks. As mentioned above, there are many works dealing with the teleoperation of industrial robots using AR technology [19,27,28,36,44], but all these approaches are far from being an alternative to conventional TP interfaces. It is also worth noting that the proposed interface is generic and applies to any commercial industrial robot.…”

Section: Features Of the Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Teleoperation of industrial robot manipulators based on augmented reality

Solanes

García

Calandin

et al. 2020

Int J Adv Manuf Technol

View full text Add to dashboard Cite

This research develops a novel teleoperation for robot manipulators based on augmented reality. The proposed interface is equipped with full capabilities in order to replace the classical teach pendant of the robot for carrying out teleoperation tasks. The proposed interface is based on an augmented reality headset for projecting computer-generated graphics onto the real environment and a gamepad to interact with the computer-generated graphics and provide robot commands. In order to demonstrate the benefits of the proposed method, several usability tests were conducted using a 6R industrial robot manipulator in order to compare the proposed interface and the conventional teach pendant interface for teleoperation tasks. In particular, the results of these usability tests show that the proposed approach is more intuitive, ergonomic and easy-to-use. Furthermore, the comparison results also show that the proposed method clearly improves the velocity of the teleoperation task, regardless of the user's previous experience in robotics and augmented reality technology.

show abstract

Section: Features Of the Proposed Methodsmentioning

confidence: 99%

“…Thus, the user was able to modify the intended goal pose of the end effector using hand gestures. A similar approach was developed in [19], where a mixed reality head-mounted display enabled the user to create and edit robot motions using waypoints.…”

Section: Related Workmentioning

confidence: 99%

Teleoperation of industrial robot manipulators based on augmented reality

Solanes

García

Calandin

et al. 2020

Int J Adv Manuf Technol

View full text Add to dashboard Cite

show abstract

“…At this stage, we can apply the simplest version of the SNS algorithm at the acceleration level [34], considering the joint acceleration limits (24), which already embed collision avoidance for the robot body, and exploiting robot redundancy to realize the other desired tasks as much as possible (see Fig. 9).…”

Section: Sns Algorithm For the First Priority Taskmentioning

confidence: 99%

“…This sensor does not need markers or specific features, allows the human to look and move freely in the workspace, and provides a sufficiently accurate pose estimation both in static and dynamic environments, during fast human motion, and in bad lighting conditions. Furthermore, it can be used to introduce a mixed reality interface for end-user robot programming [24] or for helping the operator in the quality assessment of the product of an industrial process [25].…”

Section: Introductionmentioning

confidence: 99%

Human-robot contactless collaboration with mixed reality interface

Khatib

Khudir

Luca

2021

Robotics and Computer-Integrated Manufacturing

View full text Add to dashboard Cite

A control system based on multiple sensors is proposed for the safe collaboration of a robot with a human. New constrained and contactless human-robot coordinated motion tasks are defined to control the robot end-effector so as to maintain a desired relative position to the human head while pointing at it. Simultaneously, the robot avoids any collision with the operator and with nearby static or dynamic obstacles, based on distance computations performed in the depth space of a RGB-D sensor. The various tasks are organized with priorities and executed under hard joint bounds using the Saturation in the Null Space (SNS) algorithm. A direct human-robot communication is integrated within a mixed reality interface using a stereo camera and an augmented reality system. The proposed system is significant for on-line, collaborative quality assessment phases in a manufacturing process. Various experimental validation scenarios using a 7-dof KUKA LWR4 robot are presented.

show abstract

“…ROS provides multinode and independent node support coming from a pre-built system, as well as several systems, tools, and algorithms to provide the basis for controlling complex robots [56]. Some of these tools are programming languages such as C++ and Python, to program and connect to interfaces of the robots, graphical user interfaces (GUI) such as RViz for visualisation of robot sensor data, and many others [59].…”

Section: Rosmentioning

confidence: 99%

A Prototype that Fuses Virtual Reality, Robots, and Social Networks to Create a New Cyber–Physical–Social Eco-Society System for Cultural Heritage

2020

View full text Add to dashboard Cite

With the rapid development of technology and the increasing use of social networks, many opportunities for the design and deployment of interconnected systems arise that could enable a paradigm shift in the ways we interact with cultural heritage. The project described in this paper aims to create a new type of conceptually led environment, a kind of Cyber–Physical–Social Eco-Society (CPSeS) system that would seamlessly blend the real with virtual worlds interactively using Virtual Reality, Robots, and Social Networking technologies, engendered by humans’ interactions and intentions. The project seeks to develop new methods of engaging the current generation of museum visitors, who are influenced by their exposure to modern technology such as social media, smart phones, Internet of Things, smart devices, and visual games, by providing a unique experience of exploring and interacting with real and virtual worlds simultaneously. The research envisions a system that connects visitors to events and/or objects separated either in time or in space, or both, providing social meeting points between them. To demonstrate the attributes of the proposed system, a Virtual Museum scenario has been chosen. The following pages will describe the RoboSHU: Virtual Museum prototype, its capabilities and features, and present a generic development framework that will also be applicable to other contexts and sociospatial domains.

show abstract

End-User Robot Programming Using Mixed Reality

Cited by 56 publications

References 15 publications

Teleoperation of industrial robot manipulators based on augmented reality

Teleoperation of industrial robot manipulators based on augmented reality

Human-robot contactless collaboration with mixed reality interface

A Prototype that Fuses Virtual Reality, Robots, and Social Networks to Create a New Cyber–Physical–Social Eco-Society System for Cultural Heritage

Contact Info

Product

Resources

About