A structured multi-feature representation for recognizing human action and interaction

Liu, Bangli; Ju, Zhaojie; Liu, Honghai

doi:10.1016/j.neucom.2018.08.066

Cited by 27 publications

(10 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One reason is that our method was mainly about human interaction recognition, the features of single person' actions got weakened due to the side effect of zero padding, which affected our recognition results. Hierarchical RNN [11] 59.10% 64.00% Dynamic skeletons [34] 60.23% 65.22% ST-LSTM+Trust Gate [13] 69.20% 77.70% Two-stream RNNs [24] 71.30% 79.50% STA-LSTM [30] 73.40% 81.20% Res-TCN [22] 74.30% 83.10% ST-GCN [35] 81.50% 88.30% Multiview IJTM [21] 82.96% 90.12% HCN [31] 86.50% 91.10% Proposed Method 82.53% 91.75%…”

Section: Methods Accuracymentioning

confidence: 99%

“…For the original NTU RGB+D dataset, we transposed the original coordinate system to human-centric coordinate system. Different from [30], we always chose the first person's body center as the center of the coordinate system in order to better express the relative position between two subjects. Furthermore, coordinate transformation can eliminate the influence of different perspectives of actions.…”

Section: Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-Feature Representation of Skeleton Sequences for Human Interaction Recognition

Wang

Deng

2020

Electronics

View full text Add to dashboard Cite

Inspired from the promising performances achieved by recurrent neural networks (RNN) and convolutional neural networks (CNN) in action recognition based on skeleton, this paper presents a deep network structure which combines both CNN for classification and RNN to achieve attention mechanism for human interaction recognition. Specifically, the attention module in this structure is utilized to give various levels of attention to various frames by different weights, and the CNN is employed to extract the high-level spatial and temporal information of skeleton data. These two modules seamlessly form a single network architecture. In addition, to eliminate the impact of different locations and orientations, a coordinate transformation is conducted from the original coordinate system to the human-centric coordinate system. Furthermore, three different features are extracted from the skeleton data as the inputs of three subnetworks, respectively. Eventually, these subnetworks fed with different features are fused as an integrated network. The experimental result shows the validity of the proposed approach on two widely used human interaction datasets.

show abstract

Section: Methods Accuracymentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

A Multi-Feature Representation of Skeleton Sequences for Human Interaction Recognition

Wang

Deng

2020

Electronics

View full text Add to dashboard Cite

show abstract

“…The FER task can be mainly divided into two categories depending on the type of input: static image FER and dynamic sequence FER. For the static recognition task [8]- [10], the semantic information is encoded according to a single image input, while for the dynamic recognition task [11], [12], the representation of hidden layers is related to the temporal relation among contiguous frames in the input facial expression sequence. In 2019, J Li et al [13] used Microsoft Kinect to collect the RGB-D dataset and used a two-stream network to implement a dynamic recognition task.…”

Section: A the Facial Expression Recognition Taskmentioning

confidence: 99%

A Cascaded Feature Pyramid Network With Non-Backward Propagation for Facial Expression Recognition

et al. 2021

Self Cite

View full text Add to dashboard Cite

In this work we propose a novel cascaded feature pyramid network with non-backward propagation (CFPN-NBP) for facial expression recognition (FER) that addresses the problems inherent in traditional backward propagation (BP) algorithms in the training process by using the Hilbert-Schmidt independence criterion (HSIC) bottleneck. The proposed algorithm is developed at two different levels. At the first level, a novel training method HSIC bottleneck is considered as an alternative to traditional BP optimization, where the correlation between the output of the hidden layers and the input, and the correlation between the output of the hidden layers and its label are calculated to reduce redundant information; hence, the least information is used to predict the results. At the second level, a novel architecture is designed in the feature extraction process. The convolutional layers with the same resolutions are densely connected and introduced into the attention mechanism, so that the model can focus on more important information. The convolutional layers with different resolutions are combined by three cascaded pyramid networks; in this way, the shallow features and the deep features can be further fused, and; therefore, the semantic information and the content information can both be reserved. To further reduce the number of parameters, the operation of separable convolution instead of traditional convolution is utilized. Experiments on the challenging FER2013 dataset show that the proposed CFPN-NBP algorithm improves the accuracy of the FER task and outperforms the related state-of-the-art methods.

show abstract

“…In many studies, such as [21] and [22], the selection and calibration of multiple types of sensors are considered to be key issues. The key sensors consist of industrial camera, lens, light source and image acquisition card.…”

Section: B Computer Vision Systemmentioning

confidence: 99%

Automatic and Efficient Metallic Surface Defect Detection Based on Key Pixel Point Locations

Gao

Sun

et al. 2021

IEEE Sensors J.

Self Cite

View full text Add to dashboard Cite

Surface defect detection aims to accurately recognize and distinguish types of defects and plays a key role in many applications. However, most of the recent studies focus on specific scenario detection and do not fairly consider the balance between the speed and accuracy. In the paper, we propose a key pixel points location-oriented method to identify multiscale defects, with several important properties: 1) A real-time template matching-based model is designed to speed up the process by introducing the Gaussian operator; 2) An improved Hough-based model is used to achieve a higher detection precision by deep mining both incremental properties and parallel properties; 3) An adaptive filtering-based image preprocessing method is proposed to eliminate the interference of multiple types of clutters and noises. In the experiments, a mean average rate of 96% was achieved to detect and classify four types of common defects and the average time was reduced to 0.149s. Furthermore, we fully evaluate the proposed method on two public datasets collected in real production lines and compare the results with other state-of-the-art methods. The results show that the proposed method achieved better balanced performance in many real application scenarios.

show abstract

A structured multi-feature representation for recognizing human action and interaction

Cited by 27 publications

References 49 publications

A Multi-Feature Representation of Skeleton Sequences for Human Interaction Recognition

A Multi-Feature Representation of Skeleton Sequences for Human Interaction Recognition

A Cascaded Feature Pyramid Network With Non-Backward Propagation for Facial Expression Recognition

Automatic and Efficient Metallic Surface Defect Detection Based on Key Pixel Point Locations

Contact Info

Product

Resources

About