Skeleton-based bio-inspired human activity prediction for real-time human–robot interaction

Reily, Brian; Han, Fei; Parker, Lynne E.; Zhang, Hao

doi:10.1007/s10514-017-9692-3

Cited by 29 publications

(12 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most existing skeletal features are extracted for action classification. Although Reily et al [17] proposed skeletal features for action prediction, their skeletal features cannot model the complex structure among joints in motion. The readers are referred to [18] for a systematic review of action analysis methods based on skeletal representation, respectively.…”

Section: Action Analysis In Depth Videosmentioning

confidence: 99%

“…Dynamic temporal warping [5] 54.0 Actionlet ensemble [5] (skeletal feature only) 68.0 Fourier temporal pyramid [5] 78.0 Distinctive canonical poses [24] 65.7 Relative position of joints [25] 70.0 Moving pose [26] 73.8 BIPOD representation [17] 79.7 Our approach (skeletal feature only) 85.6 Our approach 88.2 UTKinect-Action dataset HOJ3D [27] 90.9 Histogram of Direction vectors [28] 92.0 BIPOD representation [17] 92.8 Our approach (skeletal feature only) 94.3 Our approach 95.1 SYSU 3D HOI dataset ST-LSTM(Tree)+Trust Gate [29] 76.5 Part-aware LSTM [30] 76.9 BIPOD representation [17] 77.3 Our approach (skeletal feature only) 79.1 Our approach 80.7…”

Section: Msr-daily Activity Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Group Sparse Regression-Based Learning Model for Real-Time Depth-Based Human Action Prediction

Yan

Wang

2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This paper addresses the problem of predicting human actions in depth videos. Due to the complex spatiotemporal structure of human actions, it is difficult to infer ongoing human actions before they are fully executed. To handle this challenging issue, we first propose two new depth-based features called pairwise relative joint orientations (PRJOs) and depth patch motion maps (DPMMs) to represent the relative movements between each pair of joints and human-object interactions, respectively. The two proposed depth-based features are suitable for recognizing and predicting human actions in real-time fashion. Then, we propose a regression-based learning approach with a group sparsity inducing regularizer to learn action predictor based on the combination of PRJOs and DPMMs for a sparse set of joints. Experimental results on benchmark datasets have demonstrated that our proposed approach significantly outperforms existing methods for real-time human action recognition and prediction from depth data.

show abstract

Section: Action Analysis In Depth Videosmentioning

confidence: 99%

Section: Msr-daily Activity Datasetmentioning

confidence: 99%

Group Sparse Regression-Based Learning Model for Real-Time Depth-Based Human Action Prediction

Yan

Wang

2018

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Collaborative robotics, including multi-robot systems [4,7,38] and human-robot collaboration [33,37], has been widely studied over the past decades due to its effectiveness and flexibility to address large-scale collaborative tasks. Collaborative perception is a fundamental capability in collaborative robotics for robots and other agents including humans in a collaborative team to share information of the surrounding environment thus achieving shared situational awareness among the teammates.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Gao¹,

Zhang²

2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

Correspondence identification is essential for multirobot collaborative perception, which aims to identify the same objects in order to ensure consistent references of the objects by a group of robots/agents in their own fields of view. Although recent deep learning methods have shown encouraging performance on correspondence identification, they suffer from two shortcomings, including the inability to address non-covisibility in collaborative perception that is caused by occlusion and limited fields of view of the agents, and the inability to quantify and reduce uncertainty to improve correspondence identification. To address both issues, we propose a novel uncertainty-aware deep graph matching method for correspondence identification in collaborative perception. Our new approach formulates correspondence identification as a deep graph matching problem, which identifies correspondences based upon graph representations that are constructed from the agents' observations. We introduce a novel deep graph matching network under the Bayesian framework to explicitly quantify uncertainty in the identified correspondences. In addition, we design a novel loss function that explicitly reduces correspondence uncertainty and perceptual non-covisibility during learning. We evaluate our approach in the robotics applications of collaborative assembly and multi-robot coordination using high-fidelity simulations and physical robots. Experiments have shown that, through addressing both uncertainty and non-covisibility, our approach achieves the state-of-the-art performance of correspondence identification.

show abstract

“…The classic moving target detection algorithm can be divided into inter-frame difference method, background difference method, and optical flow method [15]. The background difference method is to use the information of the previous frames to establish a background model for each pixel, and determine whether each pixel belongs to a moving target through the matching degree of the current frame pixel to be detected and its background model [16]. The algorithm has a good detection effect and real-time high sex has been widely used in practical engineering.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Detection and Spatial Segmentation of Difference Image Motion Changes

et al. 2020

View full text Add to dashboard Cite

In this paper, real-time detection of moving targets is performed, and an improved moving target detection algorithm combining background modeling and three-frame difference method is studied. An improved anisotropic differential filtering algorithm is proposed. The algorithm mainly compares the gradient difference between the target and the background in eight nearby directions. Choose the average of the three directions with the smallest spread function value to filter the space, which will effectively highlight the difference between the targets. so that the target signal is well preserved in the differential space domain; then based on obtaining the difference map. The average gray level of the target and the local signal-to-noise ratio of the spatial domain obtained by the energy enhancement algorithm combining the spatial and temporal motion characteristics have reached 278 and 14.84 dB, which further improves the target signal. Strict registration of heterogeneous spatial domains is completed by affine transformation and bilinear interpolation, and in many cases, the invariance and robustness of the method are verified. Focus on the proposed color space fusion method based on spatial frequency and fuzzy transformation. The superiority of the methods is evaluated and compared from both subjective and objective aspects. Finally, it lays a good foundation for moving target detection and better dealing with complex scenes. The improved inter-frame difference method combined with background subtraction validates and compares the method in different scenarios. It shows the advanced nature of the method in both qualitative and quantitative aspects. And effectively overcome the shortcomings of the particle filter tracking algorithm based on a single feature.

show abstract

Skeleton-based bio-inspired human activity prediction for real-time human–robot interaction

Cited by 29 publications

References 60 publications

Group Sparse Regression-Based Learning Model for Real-Time Depth-Based Human Action Prediction

Group Sparse Regression-Based Learning Model for Real-Time Depth-Based Human Action Prediction

Bayesian Deep Graph Matching for Correspondence Identification in Collaborative Perception

Real-Time Detection and Spatial Segmentation of Difference Image Motion Changes

Contact Info

Product

Resources

About