Airwriting recognition using wearable motion sensors

Abstract

In this work we present a wearable input device which enables the user to input text into a computer. The text is written into the air via character gestures, like using an imaginary blackboard. To allow hands-free operation, we designed and implemented a data glove, equipped with three gyroscopes and three accelerometers to measure hand motion. Data is sent wirelessly to the computer via Bluetooth. We use HMMs for character recognition and concatenated character models for word recognition. As features we apply normalized raw sensor signals. Experiments on single character and word recognition are performed to evaluate the end-to-end system. On a character database with 10 writers, we achieve an average writer-dependent character recognition rate of 94.8% and a writer-independent character recognition rate of 81.9%. Based on a small vocabulary of 652 words, we achieve a single-writer word recognition rate of 97.5%, a performance we deem is advisable for many applications. The final system is integrated into an online word recognition demonstration system to showcase its applicability.

Feedback-based handwriting recognition from inertial sensor data for wearable devices

“…The work most related to ours is the 'Air Writing' system by Amma et.al. [21,22]. In this system, the authors used HMM based systems to directly recognize words and use a language model to correct errors in sentences.…”

Sensing Gestures for Business Intelligence

“…Magnetometers measure the ambient magnetic field and provide digital compass like applications on smartphone devices. These are also more traditional sensors which are small and come at a reasonable cost (Amma, et al, 2010) when compared to smartphones or tablets. The first generation of sensors included pen based and hand worn sensors known as 'data gloves.'…”

6D motion gesture recognition using spatio-temporal features

“…In [3], gravitational acceleration is compensated by subtracting the mean of Ao based on the assumption that the sensor heading is constant over the time of one recording. Apparently this does not work in our case, and we have to use extra information, i.e., the orientation, to remove the gravitational acceleration.…”

“…The tracking results can be either implicit or explicit, with or without the orientation information. Gesture recognition is commonly done with hidden Markov models [1,2,3]. Other approaches for gesture recognition include dynamic time warping [4], data-driven template matching [5,6], and feature-based statistical classifiers [7,8,9].…”