No abstract

No abstract

No abstract

There is an on-going study which aims to assist in teaching Sign Language to hearing impaired children by means of non-verbal communication and imitation based interaction games between a humanoid robot and the child. In this study, the robot will be able to express a word in Sign Language among a set of chosen words using hand movements, body and face gestures and having comprehended the word, the child will give relevant feedback to the robot.This study proposes an interactive game between a NAO H25 humanoid robot and preschool children based on Sign Language. Currently the demo is in Turkish Sign Language (TSL) but it will be extended to ASL, too. Since the children do not know how to read and write, and are not familiar with sign language, we prepared a short story including special words where the robot realized the specially selected word with sign language as well as pronouncing the word verbally. After realizing every special word with sign language the robot waited for response from children, where the children were asked to show colour flashcards with the illustration of the word. If the flashcard and the word match the robot pronounces the word verbally and continues to tell the story. At the end of the story the robot realizes the words one by one with sign language in a random order and asks the children to put the sticker of the relevant flashcard on their play cards which include the story with illustrations of the flashcards. We also carried the game to internet and tablet pc environments. The aim is to evaluate the children's sign language learning ability from a robot, in different embodiments and make the system available to children disregarding the cost of the robot, transportation and knowhow issues.

This paper presents results from interactive robotic games based on a multimodal user interface. This game is intended to be used as part of children's education and therapy, within a project based on human-robot and humancomputer interaction games featuring non-verbal gestures and imitation for the purpose of teaching sign language. The humanoid robot acts as a social peer and assistant in the games to motivate the child, teach non verbal signs, evaluate the child's effort, and give appropriate feedback to improve the learning and recognition rate of children. In the games, videos of the robot are used, and the performance of the children with hearing impairment are assessed in terms of subjective and objective evaluation criteria. We verified that the results of the tests using the robot's video-based games are in accordance with the physical robots performing same word sets and test setups.