Skeleton and visual tracking fusion for human following task of service robots

Babaians, Edwin; Korghond, Navid Khazaee; Ahmadi, Alireza; Karimi, Mojtaba; Ghidary, Saeed Shiry

doi:10.1109/icrom.2015.7367878

Cited by 11 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work properly employs the NiTE SDK to ascertain the location of the hand center point. 39 Figure 7 shows the estimated hand center point (i.e. the red point depicted in Figure 7) by the NiTE SDK.…”

Section: The Developed Smart Material-handling Robot System With Hri ...mentioning

confidence: 99%

“…To quickly find the hand point (if existing) location in the binary-valued image by the NiTE SDK, an initial procedure is implemented to obtain regular motion vector information between consecutive images (similar to eye detection using variants of motion vectors). In the NiTE SDK, a total of three dynamic hand gesture actions are defined to make such motion vector variants, 39 namely “hand-waving,”“hand-clicking,” and “hand-raising.” In this work, the gesture action of “hand-clicking,” representing “pushing forward and then retracting,” is used. In addition, when the hand point is well-detected, hand tracking then follows, in which keeping the hand “pushing forward” and performing the specific HRI hand gesture action will then be helpful for segmentations of the significant hand part.…”

Section: The Developed Smart Material-handling Robot System With Hri ...mentioning

confidence: 99%

Section: Data Acquisition and Preprocess Of Kinect Depth Sensor-based...mentioning

confidence: 99%

See 2 more Smart Citations

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Ding

2022

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

With rapid developments in biometric recognition, a great deal of attention is being paid to robots which interact smartly with humans and communicate certain types of biometrical information. Such human–machine interaction (HMI), also well-known as human–robot interaction (HRI), will, in the future, prove an important development when it comes to automotive manufacturing applications. Currently, hand gesture recognition-based HRI designs are being practically used in various areas of automotive manufacturing, assembly lines, supply chains, and collaborative inspection. However, very few studies are focused on material-handling robot interactions combined with hand gesture communication of the operator. The current work develops a depth sensor-based dynamic hand gesture recognition scheme for continuous-time operations with material-handling robots. The proposed approach properly employs the Kinect depth sensor to extract features of Hu moment invariants from depth data, through which feature-based template match hand gesture recognition is developed. In order to construct continuous-time robot operations using dynamic hand gestures with concatenations of a series of hand gesture actions, the wake-up reminder scheme using fingertip detection calculations is established to accurately denote the starting, ending, and switching timestamps of a series of gesture actions. To be able to perform typical template match on continuous-time dynamic hand gesture recognition with the ability of real-time recognition, representative frame estimates using centroid, middle, and middle-region voting approaches are also presented and combined with template match computations. Experimental results show that, in certain continuous-time periods, the proposed complete hand gesture recognition framework can provide a smooth operation for the material-handling robot when compared with robots controlled using only extractions of full frames; presented representative frames estimated by middle-region voting will maintain fast computations and still reach the competitive recognition accuracy of 90.8%. The method proposed in this study can facilitate the smart assembly line and human–robot collaborations in automotive manufacturing.

show abstract

Section: The Developed Smart Material-handling Robot System With Hri ...mentioning

confidence: 99%

Section: The Developed Smart Material-handling Robot System With Hri ...mentioning

confidence: 99%

Section: Data Acquisition and Preprocess Of Kinect Depth Sensor-based...mentioning

confidence: 99%

See 1 more Smart Citation

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Ding

2022

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

show abstract

“…[21] combined an omnidirectional camera and a laser to obtain robust tracking in an outdoor environment. In recent years, combined RGB image and depth sensing has enabled skeleton tracking, which can be used in combination with other visual detectors and filters for human tracking [4]. In the case of occlusion, an Extended Kalman Filter (EKF) based method has been used Ref.…”

Section: Person-following Robotmentioning

confidence: 99%

Facilitating Human-Mobile Robot Communication via Haptic Feedback and Gesture Teleoperation

Che

Culbertson

Tang

et al. 2018

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

In this article, we present a bi-directional communication scheme that facilitates interaction between a person and a mobile robot that follows the person. A person-following robot can assist people in many applications including load carrying, elder care, and emotional support. However, commercially available personal robot systems usually have limited sensing and actuation capabilities. They are not expected to function perfectly in complex environments, and human intervention is required when the robot fails. We propose to use a holdable mechatronic device to reduce the user's effort in communication and enable natural interaction during the intervention. Our design of the holdable device consists of two parts: a haptic interface that displays touch cues to convey the robot's failure status via asymmetric vibrations, and a command interface for teleoperating the robot follower with hand gestures. We experimentally evaluated the device and the communication strategy in two sets of user studies with a controlled environment and a physical robot follower. Results show that with the proposed method, users are able to perform their tasks better, respond to robot failure events faster, and adjust walking speed according to the robot's limitations. We also demonstrate that users can successfully teleoperate the robot to avoid obstacles when navigating in challenging environments.

show abstract

Optimal path planning and execution for mobile robots using genetic algorithm and adaptive fuzzy-logic control

Bakdi

Hentout

Boutami

et al. 2017

Robotics and Autonomous Systems

211

View full text Add to dashboard Cite

Skeleton and visual tracking fusion for human following task of service robots

Cited by 11 publications

References 13 publications

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Designs of human–robot interaction using depth sensor-based hand gesture communication for smart material-handling robot operations

Facilitating Human-Mobile Robot Communication via Haptic Feedback and Gesture Teleoperation

Optimal path planning and execution for mobile robots using genetic algorithm and adaptive fuzzy-logic control

Contact Info

Product

Resources

About