Hand Gesture Recognition with Inertial Sensors

Abstract

Dyscalculia is a learning difficulty hindering fundamental arithmetical competence. Children with dyscalculia often have difficulties in engaging in lessons taught with traditional teaching methods. In contrast, an educational game is an attractive alternative. Recent educational studies have shown that gestures could have a positive impact in learning. With the recent development of low cost wearable sensors, a gesture based educational game could be used as a tool to improve the learning outcomes particularly for children with dyscalculia. In this paper, two generic gesture recognition methods are proposed for developing an interactive educational game with wearable inertial sensors. The first method is a multilayered perceptron classifier based on the accelerometer and gyroscope readings to recognize hand gestures. As gyroscope is more power demanding and not all low-cost wearable device has a gyroscope, we have simplified the method using a nearest centroid classifier for classifying hand gestures with only the accelerometer readings. The method has been integrated into open-source educational games. Experimental results based on 5 subjects have demonstrated the accuracy of inertial sensor based hand gesture recognitions. The results have shown that both methods can recognize 15 different hand gestures with the accuracy over 93%.

Energy per Operation Optimization for Energy-Harvesting Wearable IoT Devices

“…Researchers have proposed sensor networks, gesture-based control, health monitoring, and activity monitoring as potential applications of IoT devices [16][17][18][19][20][21][22]. Gesture recognition using wearable devices has received significant attention due to its applications in human-computer interaction, gesture-based control, and virtual reality [23][24][25][26]. For instance, impedance sensing [25] and EMG sensors [23,26] on wearable devices are used to recognize gestures.…”

“…For instance, impedance sensing [25] and EMG sensors [23,26] on wearable devices are used to recognize gestures. The use of motion sensors with dynamic time warping (DTW) [24,27] and template matching methods [27] have also been proposed to identify gestures. While these studies achieved high recognition accuracy, most of them were implemented and tested offline on the host machines.…”

Analysis of the effectiveness of machine learning methods in the problem of gesture recognition based on the data of electromyographic signals

“…Системы распознавания жестов широко используются для реализации дистанционного управления робототехническими устройствами, а также в медицине для восстановления утерянных двигательных функций и протезирования конечностей пациентов. Для решения задачи распознавания жестов применяются электромиография, методы обработки изображений [Nagashree et al, 2005;Mehra, 2018;Nguyen et al, 2014;Lee et al, 2020;Cho et al, 2018], анализ пространственной ориентации объекта, полученной при помощи инерциальных датчиков [Teachasrisaksakul et al, 2018], эпидермальные тактильные датчики [Byun, Lee, 2019] и т. д. Для систем распознавания жестов на основе методов обработки изображений обязательным условием является расположение руки непосредственно в пределах обзора камеры (нескольких камер), что затрудняет их использование в заранее неподготовленной среде и усложняет аппаратную базу. Системы распознавания жестов, основанные на инерциальных датчиках, имеют высокую точность классификации, однако не могут быть использованы для распознавания движений пальцев [Teachasrisaksakul et al, 2018].…”