Capturing Expressive and Indicative Qualities of Conducting Gesture: An Application of Temporal Expectancy Models

A Review of Temporal Aspects of Hand Gesture Analysis Applied to Discourse Analysis and Natural Conversation

“…In this review, we found that 3D approaches are used in [7,26,36,37,41,42] through setting synchronized cameras in different positions, in order to estimate tridimensional positions. In [28], a 3D approach is also employed, but by estimating depth information through the use of markers -the marker size indicates depth information. A special case is [23], in which there are two different datasets: one obtained through a 3D vision-based approach based on Kinect; and other obtained using multitouch devices.…”

“…Corradini [6] uses a tracking method based on skin color and contour to detect hands and head positions. Swaminathan et al [28] use tracking based on markers. Choi et al [31] perform tracking through a Mean Shift Algorithm, which relies on the distribution of some visual features, such as color and texture.…”

“…Corradini [6] considers the angle between hands and head and derives other features from velocity. Swaminathan et al [28] consider velocity and direction of the movements. It is interesting to note that all studies focusing on supporting psycholinguistic analysis tasks use only features based on hand position information, such as coordinates, amount of effort, and presence of holds or symmetry.…”

“…Recognition Method [21] -Conditional Modality Fusion [22] -Bayesian Model [16] -HMM trained with Baum-Welch algorithm [6] -Hybrid architecture with RNN and HMM [24] -T-CombNet (Neural Network model) [25] -SVM, DTA and DTW; CDP [26] Recognition-based segmentation Bayesian Decision Tree [27] Recognition-based segmentation Bayesian Analysis [29] Recognition-based segmentation DTW, Nearest Neighbor [10] Recognition-based segmentation FSM [30] Recognition-based segmentation HMM trained with Baum-Welch algorithm [31] Recognition-based segmentation K-means with Longest Common Subsequence [32] Recognition-based segmentation Relevance Vector Machine [15] Recognition-based segmentation HMM [33] Recognition-based segmentation CDP [17] Recognition-based segmentation HMM [28] Recognition-based segmentation HP, Bayesian framework, and SSM [39] Threshold-based segmentation Based on hand velocity [36] Threshold-based segmentation Heuristic based on effort amount calculating [37] Threshold-based segmentation Heuristic -correlation between hand motion [35] Filter-based segmentation CWT [34] Filter-based segmentation WFT and CWT [38] Segmentation based on turning point detection Based on hand acceleration [40] Segmentation based on turning point detection HMM trained with EM [7] Recognition-based segmentation combined with turning point detection…”

“…The work [30] is similar to [10], since HMM is used in both to model gesture "phases": in [30], it is aimed at recognizing "up" and "down" movements, considering the temporal structure of orchestra conducting gestures. In [28], orchestra conducting gestures are analyzed for capturing their expressive qualities, such as "beat" and "tempo", and indicative qualities, such as "articulation". Finally, in [21], it is analyzed gesticulation to provide additional features aiming at improving coreference analysis in videos where speakers describe the behavior of a mechanical device.…”