Measuring and reducing observational latency when recognizing actions

Abstract

An important aspect in interactive, action-based interfaces is the latency in recognizing the action. High latency will cause the system's feedback to lag behind user actions, reducing the overall quality of the user experience. This paper presents a novel dataset and algorithms for reducing the latency in recognizing the action. Latency in classification is minimized with a classifier based on logistic regression that uses canonical poses to identify the action. The classifier is trained from the dataset using a learning formulation that makes it possible to train the classifier to reduce latency. The classifier is compared against both a Bag of Words and a Conditional Random Field classifier and is found to be superior in both pre-segmented and on-line classification tasks.

Two-person interaction detection using body-pose features and multiple instance learning

“…Both papers limit input to color and depth maps. Only Masood et al [19] and Sung et al [32] use joint sequences from depth sensors as a feature. In [19], only skeleton joints are used as a feature for real-time single activity recognition and actions are detected by logistic regression.…”

“…These datasets are focused on simple activities or gestures [19,16,2], or daily activities [22,32] performed by a single actor such as drinking water, cooking, entering the room, etc.…”

Joint movement similarities for robust 3D action recognition using skeletal data

“…From skeletal joint locations, Yang and Tian [27] used the relative differences of joint positions, which is a compact representation of the structure of the skeleton for actions involving multiple joints. In a number of existing approaches, the displacement of joint position between the current frame and other frames is calculated in order to capture the joint motion across time [28][29][30]. There are also some techniques associating features extracted from depth data with 3D joint data for action recognition that involves interaction with the environment [31,32].…”

“…A number of other approaches combine different types of joint features [28][29][30]32] but as all skeleton joints in each frame are involved, noise introduced by unrelated joints during specific actions may degrade the classification result. Furthermore, since they mostly use direct concatenation of relative positions of joints in combination with joint displacements across time, noise introduced in one feature will impact the discriminative power of the entire feature vector.…”

Space-time representation of people based on 3D skeletal data: A review

“…Representations based directly on raw joint positions are widely used due to the simple acquisition from sensors. Although normalization procedures can make human representations partially invariant to view and scale variations, more [210] Moving Pose BoW Lowlv Dict Bloom et al [211] Dynamic Features Conc Lowlv Hand Vemulapalli et al [212] Lie Group Manifold Conc Manif Hand Zhang and Parker [213] BIPOD Stat Body Hand Lv and Nevatia [214] HMM/Adaboost Conc Lowlv Hand Herda et al [215] Quaternions Conc Body Hand Negin et al [216] RDF Kinematic Features Conc Lowlv Unsup Masood et al [217] Logistic Regression Conc Lowlv Hand Meshry et al [218] Angle & Moving Pose BoW Lowlv Unsup Tao and Vidal [219] Moving Poselets BoW Body Dict Eweiwi et al [220] Discriminative Action Features Conc Lowlv Unsup Wang et al [221] Ker-RP Stat Lowlv Hand Salakhutdinov et al [222] HD Models Conc Lowlv Deep sophisticated construction techniques (e.g., deep learning) are typically needed to develop robust human representations. Representations without involving temporal information are suitable to address problems such as pose and gesture recognition.…”