Adaptive Gesture Recognition with Variation Estimation for Interactive Systems

Abstract

This paper presents a gesture recognition/adaptation system for Human Computer Interaction applications that goes beyond activity classification and that, complementary to gesture labeling, characterizes the movement execution. We describe a template-based recognition method that simultaneously aligns the input gesture to the templates using a Sequential Montecarlo inference technique. Contrary to standard templatebased methods based on dynamic programming, such as Dynamic Time Warping, the algorithm has an adaptation process that tracks gesture variation in real-time. The method continuously updates, during execution of the gesture, the estimated parameters and recognition results which offers key advantages for continuous human-machine interaction. The technique is evaluated in several different ways: recognition and early recognition are evaluated on a 2D onscreen pen gestures; adaptation is assessed on synthetic data; and both early recognition and adaptation is evaluation in a user study involving 3D free space gestures. The method is not only robust to noise and successfully adapts to parameter variation but also performs recognition as well or better than non-adapting offline template-based methods.

Accurate online alignment of human motor performances

“…Some approaches such as [Van Hanh et al 2009] and [Caramiaux et al 2015], which aim at the alignment of motion capture data, rely on algorithms other than DTW, partly to be less prone to noise and outliers. However, these two related approaches rely on parameters that have to be adjusted manually whereas our implementation does not have such critical parameters that directly affect alignment performance.…”

User-Centred Design Actions for Lightweight Evaluation of an Interactive Machine Learning Toolkit

“…The beta release included several new features; additional learning algorithms, such as DTW, Gaussian Mixture Models (GMM), Hierarchical Hidden Markov Models (HHMM)-via improved integration with the XMM package (Françoise, Schnell, & Bevilacqua, 2013)-and particle filtering-via integration with Gesture Variation Follower (Caramiaux, Montecchio, Tanaka, & Bevilacqua, 2014); a new class library with signal processing primitives (e.g., circular buffer, Root Mean Square, Mel Frequency Cepstral Coefficients, first-and second-order derivatives, etc. ), and an improved web API for cloud-based multimodal data storage and retrieval.…”

Body, Space, and Emotion: A Perceptual Study

“…In this context, the lack of systematic coding has been successfully compensated for through a promising alternative approach developed by Caramiaux (2014). Caramiaux, Montecchio, Tanaka and Bevilacqua (2014) investigated and demonstrated how the variability of the body behavior itself can stand as a central cue for capturing expressivity. However, other than the seminal work by Pollick, Paterson, Bruderlin, and Sanford (2001) on motion of knocking, very few experiments have investigated emotional communication through specific limb variations.…”