Gesture Interactions With Video: From Algorithms to User Evaluation

Abstract

In the context of immersive communications that aim to enable natural experiences and interactions among people, objects, and the environment, we propose a method to enable natural video interactions through hand gesture recognition between users and a video meeting system. An end‐to‐end study was performed: we started with the development of specific gesture recognition algorithms and concluded with a user evaluation to validate our results. Gestures and their associated functionalities were identified via a user survey which focused on distinguishing two concepts which are often confused: hand posture and hand gesture (i.e., static versus dynamic). Our recognition process was composed of two main tasks: hand posture recognition (i.e., skin segmentation, background subtraction, region combination, feature extraction, and classification) and hand gesture recognition (tracking and recognition). Our approach combined a signal similarity study with a data‐mining tool for dynamic gesture recognition. We focused on the experimentation and user evaluation to improve our approach, taking into account user feedback and analyzing performance in different environments and for different users. © 2013 Alcatel‐Lucent.

A Dynamic Hand Gesture Recognition System Incorporating Orientation-based Linear Extrapolation Predictor and Velocity-assisted Longest Common Subsequence Algorithm

“…As shown in Fig. 16, in terms of deviation, the proposed method has the best performance in index of 2,3,6,7,8,9,11,12,13,14, and the average deviation of the proposed method in Fig. 16 is 3.374 pixels, while the average deviations by uniform velocity predictor and Kalman filter are 4.174 pixels and 6.257 pixels, respectively.…”

“…To evaluate the tracking accuracy of the proposed method, we compare the actual trajectory with three predicted trajectories determined by three different predictors, including uniform velocity predictor employed in [14], Kalman filter predictor employed in [13] and the proposed predictor. Fig.…”

“…Due to a relatively high computational complexity, it is difficult to meet the requirements of real time tracking in dynamic gesture recognition system. Kalman filter was one of the widely used tracking algorithms [12,13], and it established the state model of system by maximizing the posteriori probability of history measurements to forecast the target state. Traditional linear extrapolation method was employed in [14,15] with the assumption that the hands always maintain uniform linear motion on both horizontal and vertical directions in each neighboring frames.…”

Visual rendering of shapes on 2D display devices guided by hand gestures

“…. Q T of length T, we calculate the probability of observation or likelihood as given in (30), where π q 1 is initial probability of state 1.…”

HD: Efficient Hand Detection and Tracking

“…Among major colour spaces such as RGB [34], HSV [35], or YIQ [36], our system uses YCbCr colour space for thresholding to distinguish between skin and non-skin colours. Indeed, the YCbCr colour space is the most popular choice in skin-colour detection methods, because its luminance component Y as well as chrominance components Cb and Cr are separated and could be easily computed from RGB values [37].…”