Inertial Sensor-Based Touch and Shake Metaphor for Expressive Control of 3D Virtual Avatars

Abstract

In this paper, we present an inertial sensor-based touch and shake metaphor for expressive control of a 3D virtual avatar in a virtual environment. An intuitive six degrees-of-freedom wireless inertial motion sensor is used as a gesture and motion control input device with a sensor fusion algorithm. The algorithm enables user hand motions to be tracked in 3D space via magnetic, angular rate, and gravity sensors. A quaternion-based complementary filter is implemented to reduce noise and drift. An algorithm based on dynamic time-warping is developed for efficient recognition of dynamic hand gestures with real-time automatic hand gesture segmentation. Our approach enables the recognition of gestures and estimates gesture variations for continuous interaction. We demonstrate the gesture expressivity using an interactive flexible gesture mapping interface for authoring and controlling a 3D virtual avatar and its motion by tracking user dynamic hand gestures. This synthesizes stylistic variations in a 3D virtual avatar, producing motions that are not present in the motion database using hand gesture sequences from a single inertial motion sensor.

Handwriting Recognition in Free Space Using WIMU-Based Hand Motion Analysis

“…However, HMMs require large training data sets to form a statistical model for recognition, and their computational complexity increases with an increase in the dimensions of the feature vectors. The DTW-based hand gesture recognition algorithm has also been used by many researchers [25][26][27][28]. DTW works even with only one training data set, and it is easy to execute, computationally efficient, and accurate for time-series data.…”

“…The WIMU motion sensor is programmed with a sensor fusion algorithm that enables it to precisely and accurately track user hand motions in 3D space. A quaternion complementary filter algorithm is implemented to obtain the 3D attitude of the device in quaternion format [28]. The quaternion complementary filter algorithm uses the calibrated sensor data as input and produces the quaternion as output.…”

IMU Sensor-Based Hand Gesture Recognition for Human-Machine Interfaces

“…This more accurate distance has led to DTW being used in various fields, and, when employed for HGR, it has exhibited superior performance according to many previous studies [11,12,13,14]. However, because DTW-based HGR involves a complex learning algorithm, its applications are limited by the number of recognizable hand gestures.…”

“…Dynamic time warping (DTW) [11,12,13,14], multilayer perceptrons (MLPs) [7,15] and convolutional neural networks (CNNs) [16,17] are widely used to recognize hand gestures with IMU sensors. DTW-based recognition algorithms output the most similar hand gestures by measuring DTW distance between the input data and the representative data of each hand gesture, called templates.…”