A Gesture-Based Teleoperation System for Compliant Robot Motion

Zhang, Wei; Cheng, Hongtai; Zhao, Liang; Hao, Lina; Tao, Manli; Xiang, Chaoqun

doi:10.3390/app9245290

Cited by 22 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The current systems that utilize gestures to communicate with a human mainly focus on the teleoperation of the robot (e.g., [5]) or use gestures to command the robot without the option to connect the human gestures to the current state of the scene (e.g., [6], [7], [8]). Zhang et al [5] uses similarly to us Leap motion sensor to detect hand movements and classify individual gestures. A basic set of gestures is used to teleoperate the robotic arm without connection to the given scene.…”

Section: Related Workmentioning

confidence: 99%

Communicating human intent to a robotic companion by multi-type gesture sentences

Petr¹,

Behrens²,

Štěpánová³

et al. 2023

Preprint

View full text Add to dashboard Cite

Human-Robot collaboration in home and industrial workspaces is on the rise. However, the communication between robots and humans is a bottleneck. Although people use a combination of different types of gestures to complement speech, only a few robotic systems utilize gestures for communication. In this paper, we propose a gesture pseudolanguage and show how multiple types of gestures can be combined to express human intent to a robot (i.e., expressing both the desired action and its parameters -e.g., pointing to an object and showing that the object should be emptied into a bowl). The demonstrated gestures and the perceived tabletop scene (object poses detected by CosyPose) are processed in real-time) to extract the human's intent. We utilize behavior trees to generate reactive robot behavior that handles various possible states of the world (e.g., a drawer has to be opened before an object is placed into it) and recovers from errors (e.g., when the scene changes). Furthermore, our system enables switching between direct teleoperation of the end-effector and high-level operation using the proposed gesture sentences. The system is evaluated on increasingly complex tasks using a real 7-DoF Franka Emika Panda manipulator. Controlling the robot via action gestures lowered the execution time by up to 60%, compared to direct teleoperation.

show abstract

Section: Related Workmentioning

confidence: 99%

Communicating human intent to a robotic companion by multi-type gesture sentences

Petr¹,

Behrens²,

Štěpánová³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The issue of motion control is critical in teleoperation robotic systems. To overcome the limitations of human motion accuracy, a paper [18] introduces new interaction logic, a scalable human-robot motion mapping mechanism and a single axis mode to balance teleoperation efficiency and accuracy. In order to meet the requirements of complaint assembly skill, a vibration-based force feedback system was developed to let the operator feel the contact force.…”

Section: Motion Controlmentioning

confidence: 99%

Optimization of Motion Planning and Control for Automatic Machines, Robots and Multibody Systems

Boscariol¹,

Richiedei²

2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Teleoperation control based on body sensors has the characteristics of stability and flexibility, which can better simulate human movement and meet the needs of contactless service under epidemic conditions [ 9 , 10 , 11 , 12 ]. Among the many signal sources available, the motion of the arm is an accurate and stable signal source for hand tracking and arm movement [ 13 ].…”

Section: Introductionmentioning

confidence: 99%

Sensor Fusion-Based Teleoperation Control of Anthropomorphic Robotic Arm

et al. 2023

View full text Add to dashboard Cite

Sensor fusion is a technique that combines information from multiple sensors in order to improve the accuracy and reliability of the data being collected. In the context of teleoperation control of an anthropomorphic robotic arm, sensor fusion technology can be used to enhance the precise control of anthropomorphic robotic arms by combining data from multiple sensors, such as cameras, data gloves, force sensors, etc. By fusing and processing this sensing information, it can enable real-time control of anthropomorphic robotic arms and dexterous hands, replicating the motion of human manipulators. In this paper, we present a sensor fusion-based teleoperation control system for the anthropomorphic robotic arm and dexterous hand, which utilizes a filter to fuse data from multiple sensors in real-time. As such, the real-time perceived human arms motion posture information is analyzed and processed, and wireless communication is used to intelligently and flexibly control the anthropomorphic robotic arm and dexterous hand. Finally, the user is able to manage the anthropomorphic operation function in a stable and reliable manner. We also discussed the implementation and experimental evaluation of the system, showing that it is able to achieve improved performance and stability compared to traditional teleoperation control methods.

show abstract

A Gesture-Based Teleoperation System for Compliant Robot Motion

Cited by 22 publications

References 29 publications

Communicating human intent to a robotic companion by multi-type gesture sentences

Communicating human intent to a robotic companion by multi-type gesture sentences

Optimization of Motion Planning and Control for Automatic Machines, Robots and Multibody Systems

Sensor Fusion-Based Teleoperation Control of Anthropomorphic Robotic Arm

Contact Info

Product

Resources

About