Deep Learning Based Air-Writing Recognition with the Choice of Proper Interpolation Technique

Abstract

The act of writing letters or words in free space with body movements is known as air-writing. Air-writing recognition is a special case of gesture recognition in which gestures correspond to characters and digits written in the air. Air-writing, unlike general gestures, does not require the memorization of predefined special gesture patterns. Rather, it is sensitive to the subject and language of interest. Traditional air-writing requires an extra device containing sensor(s), while the wide adoption of smart-bands eliminates the requirement of the extra device. Therefore, air-writing recognition systems are becoming more flexible day by day. However, the variability of signal duration is a key problem in developing an air-writing recognition model. Inconsistent signal duration is obvious due to the nature of the writing and data-recording process. To make the signals consistent in length, researchers attempted various strategies including padding and truncating, but these procedures result in significant data loss. Interpolation is a statistical technique that can be employed for time-series signals to ensure minimum data loss. In this paper, we extensively investigated different interpolation techniques on seven publicly available air-writing datasets and developed a method to recognize air-written characters using a 2D-CNN model. In both user-dependent and user-independent principles, our method outperformed all the state-of-the-art methods by a clear margin for all datasets.

ImAiR: Airwriting Recognition Framework Using Image Representation of IMU Signals

“…However, these solutions require the user to carry an additional physical device, which may be inconvenient for the users. To counteract this, the second category of techniques make use of wearable gadgets such as a ring worn on the index finger [12] and smartbands [13], [14], [15] to recognize airwriting.…”

“…A convolutional neural network (CNN)-based user-independent framework was proposed in [13], for detecting airwritten English uppercase alphabets. In [15], the authors explored several techniques for interpolating signals to make them of fix-length in addition to a 2-D-CNN-based classification scheme. All these studies make use of single stage classification with cross-entropy loss minimization.…”

ImAiR: Airwriting Recognition framework using Image Representation of IMU Signals

“…However, these solutions require the user to carry an additional physical device, which may be inconvenient for the users. To counteract this, the second category of techniques make use of wearable gadgets such as a ring worn on the index finger [12] and smartbands [13]- [15] to recognize airwriting.…”

“…A Convolutional Neural Network (CNN) based user-independent framework was proposed in [13], for detecting airwritten English uppercase alphabets. In [15], the authors explored several techniques for interpolating signals to make them of fix-length in addition to a 2D-CNN based classification scheme. All these studies make use of single stage classification with cross-entropy loss minimization.…”

Handwriting Recognition Based on 3D Accelerometer Data by Deep Learning

“…Trajectory-Based Air Writing [32] Trajectory of handwriting with fingertip using a video camera and recognition was completed by a CNN and an LSTM Ten digits written with a predefined stroke ordering Air Writing with Interpolation [33] Motion sensor data are interpolated and then recognized by a 2D-CNN Uses datasets of others…”