A Human-Robot Interaction System Based on Calling Hand Gestures

Phyo, Aye Su; Fukuda, Hisato; Lam, Antony; Kobayashi, Yoshinori; Kuno, Yoshinori

doi:10.1007/978-3-030-26766-7_5

Cited by 3 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mehlmann et al [62] used Nao robot which can set up a mutual gaze and respond to verbal cues with the gaze. Phyo et al [63] highlighted on nonverbal action of people where the robot examines whether the individual is beckoning the robot by shaking hand. All of these previous HRI studies concentrated on developing simplex eye contact schemes where the robot should begin the eye contact event.…”

Section: A Related Workmentioning

confidence: 99%

Development of Duplex Eye Contact Framework for Human-Robot Inter Communication

et al. 2021

View full text Add to dashboard Cite

Establishing eye contact is the fundamental key to begin any interaction between human-human and robot-human. Two approaches are available to develop an eye contact mechanism for robot-human interaction, such as simplex and duplex. The two most critical tasks: gaze crossing and gaze awareness, are prerequisite to implementing an active eye contact mechanism in any approach. However, most past robot-human interaction studies implemented a gaze crossing function to develop eye contact in the simplex mode where a robot holds for the human to initiate the communication. However, implementing gaze crossing alone is inadequate to create an active eye contact episode; the gaze awareness function also essential to achieve. This paper aims to develop a mechanism of duplex eye contact for robot-human inter-communication satisfying both functions. This work proposes a conceptual model of a duplex eye contact mechanism considering two cases: human initiative (where the human starts communication with the robot) and robot initiative (where the robot starts the communication with the participant) to achieve a duplex eye contact mechanism. Moreover, a simple robotic system is developed consisting of four software constituents: face detection module, gaze detection and tracking module, gaze awareness module, and robot response and control module to implement the conceptual model of duplex eye contact. Several preliminary experiments are performed to extract necessary cues for designing the duplex eye contact mechanism's behavioural protocol and present their results to show the usefulness of extracted cues. Moreover, the robotic framework results in a scenario (e.g., reading the book) with the proposed duplex eye contact mechanism are presented. The results show that the proposed scheme achieved 92% and 86% accuracy for human initiative case and robot initiative case, respectively in making eye contact.

show abstract

Section: A Related Workmentioning

confidence: 99%

Development of Duplex Eye Contact Framework for Human-Robot Inter Communication

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Even an inexpensive USB camera can successfully recognize and track the movements of the user's hand and allow you to control simple actions [8]. Phyo et al demonstrated interaction with the NAO humanoid robot assistant using static hand gestures [9]. Gao et al presented gesture-based intelligent wheelchair control for the elderly and disabled, which was successfully tested in a closed environment [10].…”

Section: Introductionmentioning

confidence: 99%

Real-time robotic hand control using human gesture recognition

Egipko

Voronin

Gapon

et al. 2023

Real-Time Image Processing and Deep Learning 2023

View full text Add to dashboard Cite

Currently, there are many options for controlling robotic devices. Human-machine interaction is a key component of the control infrastructure. The most common solution is mobile devices or embedded touch screens, as well as next generation virtual reality devices. In human-machine interaction, most input devices are controlled manually, which is not always convenient, and sometimes even impossible. One option is gesture control, which has become increasingly common in the last few years. This artificial cognitive "sensory perception" or ability is a communication channel between a human and a machine. This article presents a two-steps approach to real-time control robotic devices. The first step is the hand recognition method base on palm detection (SSD Detector) and hand landmark models. After a palm detection, the hand landmark model performs fine localization of the key points of the 3-D coordinate of the hand inside the detected areas of the hand through regression and direct coordinate prediction. The model learns a consistent internal representation of the hand posture and is resistant to even partially visible hands and self-occlusions. The second human gesture recognition step is based on obtaining the coordinates of the hand, the distance from the camera to the hand in space, raised, lowered fingers and other indicators that allow you to accurately determine the shown gesture. In terms of gesture recognition accuracy, the proposed real-time system is better than the state-of-the-art methods.

show abstract

“…Phyo et.al. demonstrated an interaction with a humanoid NAO robot assistant with static hand gestures [28]. Gao et.al.…”

Section: Introductionmentioning

confidence: 99%

Pilot Studies on Avrora Unior Car-Like Robot Control Using Gestures

Nikiforov

Tsoy

Safin

et al. 2021

Smart Innovation, Systems and Technologies

View full text Add to dashboard Cite

Gesture recognition is not only an important communication channel in human-human interaction but it also allows a human to communicate with other intelligent devices. This paper presents a concept for controlling the car-like robot Avrora Unior locomotion using gestures. We created a list of 18 control commands that contains basic and compound commands. A group of 17 volunteers used this list to create individual control gestures independently. A small part of the obtained dataset of gestures was used with the Teachable machine service in order to preliminary evaluate a possibility of constructing a full-scale model and to train it appropriately.

show abstract

A Human-Robot Interaction System Based on Calling Hand Gestures

Cited by 3 publications

References 18 publications

Development of Duplex Eye Contact Framework for Human-Robot Inter Communication

Development of Duplex Eye Contact Framework for Human-Robot Inter Communication

Real-time robotic hand control using human gesture recognition

Pilot Studies on Avrora Unior Car-Like Robot Control Using Gestures

Contact Info

Product

Resources

About