Fusion of Skeletal and Silhouette-Based Features for Human Action Recognition with RGB-D Devices

Chaaraoui, Alexandros André; Padilla-López, José Ramón; Flórez-Revuelta, Francisco

doi:10.1109/iccvw.2013.19

Cited by 81 publications

(48 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ground truth kinematic structure was generated by manually connecting the centres considering the topology and kinematics. To encourage the use of our novel approach we release our code along with the new dataset 2 . All experiments were performed using a PC with an Intel Core i7-4770 CPU @ 3.40GHz (x8) and 32GB of RAM.…”

Section: Methodsmentioning

confidence: 99%

“…Kinematic structures contain skeleton information, and also provide motion related information between body parts. This information is beneficial to many higher level tasks such as human action recognition [2], body scheme learning for robotic manipulators [3], articulated objects kinematics recognition and manipulation [4], and finding kinematic correspondences between articulated objects [5]. In this paper, we focus on building the articulated kinematic structure from data, in particular, RGB image sequences with interest points tracked over time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Highly Articulated Kinematic Structure Estimation Combining Motion and Skeleton Information

Chang

Demiris

2018

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-In this paper, we present a novel framework for unsupervised kinematic structure learning of complex articulated objects from a single-view 2D image sequence. In contrast to prior motion-based methods, which estimate relatively simple articulations, our method can generate arbitrarily complex kinematic structures with skeletal topology via a successive iterative merging strategy. The iterative merge process is guided by a density weighted skeleton map which is generated from a novel object boundary generation method from sparse 2D feature points. Our main contributions can be summarised as follows: (i) An unsupervised complex articulated kinematic structure estimation method that combines motion segments with skeleton information. (ii) An iterative fine-to-coarse merging strategy for adaptive motion segmentation and structural topology embedding. (iii) A skeleton estimation method based on a novel silhouette boundary generation from sparse feature points using an adaptive model selection method. (iv) A new highly articulated object dataset with ground truth annotation. We have verified the effectiveness of our proposed method in terms of computational time and estimation accuracy through rigorous experiments with multiple datasets. Our experiments show that the proposed method outperforms state-of-the-art methods both quantitatively and qualitatively.Index Terms-Highly articulated kinematic structure estimation, adaptive motion segmentation, density weighted silhouette generation from sparse points, adaptive kernel selection.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Highly Articulated Kinematic Structure Estimation Combining Motion and Skeleton Information

Chang

Demiris

2018

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Such accurate and efficient estimation of kinematic correspondences of heterogeneous objects is beneficial in the computer vision and robotics fields. Some application areas are learning by imitation [4], human motion retargeting to robots [5], [6], human action recognition from different sensors [7], behaviour discovery and alignment [8], affordance based object/tool categorisation [9], body scheme learning for robotic manipulators [10], and articulated object manipulation [11], [12]. Although our framework can be applied to generic objects, we mainly focus on sequences containing humans and various robots as they are involved in the aforementioned applications (see Fig.…”

Section: Introductionmentioning

confidence: 99%

Learning Kinematic Structure Correspondences Using Multi-Order Similarities

Chang

Fischer

Petit

et al. 2018

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-In this paper, we present a novel framework for finding the kinematic structure correspondences between two articulated objects in videos via hypergraph matching. In contrast to appearance and graph alignment based matching methods, which have been applied among two similar static images, the proposed method finds correspondences between two dynamic kinematic structures of heterogeneous objects in videos. Thus our method allows matching the structure of objects which have similar topologies or motions, or a combination of the two. Our main contributions can be summarised as follows: (i) casting the kinematic structure correspondence problem into a hypergraph matching problem by incorporating multi-order similarities with normalising weights, (ii) introducing a structural topology similarity measure by aggregating topology constrained subgraph isomorphisms, (iii) measuring kinematic correlations between pairwise nodes, and (iv) proposing a combinatorial local motion similarity measure using geodesic distance on the Riemannian manifold. We demonstrate the robustness and accuracy of our method through a number of experiments on synthetic and real data, outperforming various other state of the art methods. Our method is not limited to a specific application nor sensor, and can be used as building block in applications such as action recognition, human motion retargeting to robots, and articulated object manipulation amongst others.

show abstract

“…[67] and can be defined as temporal pyramid of key poses. It exploits the bag of key poses model [68] and it adopts a temporal pyramid to model the temporal structure of the key poses constituting an action sequence.…”

Section: Human Action Recognition Based On Temporal Pyramid Of Key Posesmentioning

confidence: 99%

Human Action Recognition with RGB-D Sensors

Cippitelli¹,

Gambi²,

Spinsante³

2017

Motion Tracking and Gesture Recognition

View full text Add to dashboard Cite

Human action recognition, also known as HAR, is at the foundation of many different applications related to behavioral analysis, surveillance, and safety, thus it has been a very active research area in the last years. The release of inexpensive RGB-D sensors fostered researchers working in this field because depth data simplify the processing of visual data that could be otherwise difficult using classic RGB devices. Furthermore, the availability of depth data allows to implement solutions that are unobtrusive and privacy preserving with respect to classic video-based analysis. In this scenario, the aim of this chapter is to review the most salient techniques for HAR based on depth signal processing, providing some details on a specific method based on temporal pyramid of key poses, evaluated on the well-known MSR Action3D dataset.

show abstract

Fusion of Skeletal and Silhouette-Based Features for Human Action Recognition with RGB-D Devices

Cited by 81 publications

References 23 publications

Highly Articulated Kinematic Structure Estimation Combining Motion and Skeleton Information

Highly Articulated Kinematic Structure Estimation Combining Motion and Skeleton Information

Learning Kinematic Structure Correspondences Using Multi-Order Similarities

Human Action Recognition with RGB-D Sensors

Contact Info

Product

Resources

About