Early childhood education by hand gesture recognition using a smartphone based robot

Abstract

We propose a light and fast hand gesture recognition method using geometric feature for a smartphone based robot and apply it to early childhood mathematics education. The feature of hand gesture is defined by the number of extrema in the plot for the distances between the center point of hand and the outer points of hand from active contour model or snakes. The region of interest (ROI) is continuously updated by Continuously Adaptive Mean Shift Algorithm (CamShift) algorithm, and the snake model is used to make the outer points sequential efficiently. A mathematics learning application for an Android OS smartphone based robot is developed using the hand gesture recognition algorithm. The experiment with Korean children (5-6 years of age) is conducted to evaluate if hand gesture based HRI could promote their mathematics learning. The result suggests that the idea of hand gesture based HRI for early childhood education is feasible and that children can learn mathematics by hand gesture based interaction with a robot. I. INTRODUCTIONRobot technologies have been increasingly applied in education and therapy. The rapid development of HRI technology has provided more possible robot applications and has helped those applications more realistic. For example, vision based HRI technology provided more sophisticated information about human behavior that is necessary for implementation of more feasible applications [1-2], and there have been many different approaches for non-verbal communication interaction for human and robot [3].Recently, smartphones have been ubiquitous parts in our daily life, and so our lives are more connected to things that could be intelligently interactive with human or robots. Moreover, smartphones themselves have a variety of basic sensors such as cameras, inertia sensors and microphones that could be useful for HRI. Especially, various HRI approaches using 2D images were introduced for mobile phone environment [4-10].While there are many different kinds of applications for smartphone based HRI, early childhood education could be one of them. Many studies showed that cross-media and

A Novel Hybrid Deep Learning Architecture for Dynamic Hand Gesture Recognition

“…The application of gesture recognition affects many relevant research areas such as smart home devices [3], gamification [4], education [5], driving, and even lie detection [6].…”

Intelligent textiles: designing a gesture-controlled illuminated textile based on computer vision

“…Gestures play an important role in HCI by controlling the computer and machine system via specific hand movements 11 . In recent years, gesture recognition systems have been used in a range of virtual reality (VR) and augmented reality (AR) applications, including gaming 12,13 , rehabilitation [14][15][16] , health care monitoring [17][18][19] , education 20,21 and personal identification 22,23 .…”

“…Gestures play an important role in HCI by controlling the computer and machine system via specific hand movements. 11 In recent years, gesture recognition systems have been used in a range of virtual reality (VR) and augmented reality (AR) applications, including gaming, 12,13 rehabilitation, [14][15][16] health care monitoring, [17][18][19] education 20,21 and personal identification. 22,23 With the increasing interest in interdisciplinary studies in the development of e-textiles, researchers and practitioners have developed methods, techniques and materials that enable electronic systems to be embedded in soft materials.…”