Dynamical Regularity for Action Analysis

Abstract

An activity can be seen as a resultant of coordinated movement of body joints and their respective interdependencies to achieve a goal-directed task. This idea is further supported by Johansson's demonstrations that visual perception of the entire human body motion can be represented by a few bright spots which holistically describe the motion of important joints. Traditional dynamical modeling approaches usually operate on the level of individual joints of the human body, lacking any information about the interdependencies between joints. We propose a novel approach for dynamical modeling by extending conventional ideas to quantify the interdependencies between body joints. Towards this end, we propose a new approach -approximate entropy-based feature representation to model the dynamics in human movement by quantifying dynamical regularity. In this paper, we utilize the algorithmic framework of [3] for estimating approximate entropy from time series data and extend it to model the dynamics in human activities for applications such as temporal segmentation and fine-grained quality assessment of actions.Approximate entropy is a statistical tool proposed by Pincus [3, 4] for quantification of regularity of time series data and system complexity, based on the log-likelihood of repetitions of patterns of length m being close within a defined tolerance window that will exhibit similar characteristics as patterns of length (m+1) [2,3]. It assigns a non-negative number to time series data, with lower values for predictable (ordered) signals and higher values for signals with increased irregularity (or randomness). It is defined using three parameters: embedding dimension (m), radius (r), and time delay (τ). Here, m represents the length of pattern (also called as embedding vector) in the data which is checked for repeatability, τ is selected so that the components of the embedding vector are sufficiently independent, and r is used for the estimation of local probabilities. Given N data samples {x 1 , x 2 , . . . , x N }, we can define embedding vector x(i) as,The frequency of repeatable patterns of the embedding vector within a tolerance r is given by C m i (r) asApproximate Entropy is given bywhere:C m i (r) represents the frequency of repeatable patterns in the embedding vector x(i), Θ(a) is the Heaviside step function, and Φ m (r) represents the conditional frequency estimates.Multivariate Cross Approximate Entropy (XAPEN): Recent theoretical and empirical findings have demonstrated that multivariate embedding of time series data by simple concatenation of individual univariate embedding vectors achieves good state space reconstruction as evaluated by the shape and dynamics distortion measures. In this work, we propose to use the multivariate embedding procedure as described by Cao et al.[1] per body joint and estimate the approximate entropy feature representation. In addition, natural human movement involves multiple body joints interacting with each other to together accomplish a particular action task. Hence, i...

Automated surgical skill assessment in RMIS training

“…Therefore, a measure of predictability in temporal kinematic data can potentially help differentiate between varying skill levels. Approximate entropy is a measure of predictability in a time series data [20] and has been used in recent literature for activity assessment [15,8]. We extract ApEn features from our robot kinematic time series data as presented in [8].…”

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment

“…However, since their method relied solely on pose features, it neglected important visual quality cues, like splash in the case of Diving. Since accurate pose is especially difficult in sports scenarios where athletes undergo extremely convoluted poses, Venkataraman et al [25] better encoded using the approximate entropy of the poses to improve the results.…”

“…Answering these questions involves the quantification of the quality of the action -determining how well the action was carried out, also known as action quality assessment (AQA). Existing AQA [18,16,26,13,25] and skills assessment [4,10,31,32,33] approaches use a single label, known as a final score or skill-level, to train the system using some kind of regression or ranking loss function. However, the performance of these systems is limited and it seems that a single score is not sufficient to characterize a complicated action.…”

Video and accelerometer-based motion analysis for automated surgical skills assessment

“…We used K ∈ [2,3,4,5,6,7,8,9,10,12,14,16,18,20] for k-means clustering to learn motion classes (the number of time series dimensions used) for analysis of video data. The accelerometer data, however, did not have this dependency with a 6-dimensional time series (concatenation of 3-dimensional time series from two accelerometers used) for all evaluations.…”

“…In [13], the authors presented an approach of using pose with frequency features to predict sports scores. More recently, [14] used entropy features with pose to predict scores for Olympic diving videos. We take inspiration from their work and propose to encode predictability in surgical motions via entropy based features for skills assessment.…”