Weighted score-level feature fusion based on Dempster–Shafer evidence theory for action recognition

Zhang, Guoliang; Jia, Songmin; Li, Xiuzhi; Zhang, Xiangyin

doi:10.1117/1.jei.27.1.013021

Cited by 5 publications

(9 citation statements)

References 51 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of video activity prediction on the UT-Interaction dataset were compared with those of Hierarchical-M [19], TGTW [17], HSOM [13], Dynamic BOW [15], multi-stage LSTM [18], P-TS [21], and RU-LSTM [24]. The results of video activity prediction on the sub-JHMDB dataset were compared with those of PDP [14], confidence-DT [33], WSF-DS [34], RPAN [16], DRN [22], and RGN [23].…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Human activity prediction using saliency-aware motion enhancement and weighted LSTM network

Weng

Zhang

2021

J Image Video Proc.

View full text Add to dashboard Cite

In recent years, great progress has been made in recognizing human activities in complete image sequences. However, predicting human activity earlier in a video is still a challenging task. In this paper, a novel framework named weighted long short-term memory network (WLSTM) with saliency-aware motion enhancement (SME) is proposed for video activity prediction. First, a boundary-prior based motion segmentation method is introduced to use shortest geodesic distance in an undirected weighted graph. Next, a dynamic contrast segmentation strategy is proposed to segment the moving object in a complex environment. Then, the SME is constructed to enhance the moving object by suppressing irrelevant background in each frame. Moreover, an effective long-range attention mechanism is designed to further deal with the long-term dependency of complex non-periodic activities by automatically focusing more on the semantic critical frames instead of processing all sampled frames equally. Thus, the learned weights can highlight the discriminative frames and reduce the temporal redundancy. Finally, we evaluate our framework on the UT-Interaction and sub-JHMDB datasets. The experimental results show that WLSTM with SME statistically outperforms a number of state-of-the-art methods on both datasets.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Human activity prediction using saliency-aware motion enhancement and weighted LSTM network

Weng

Zhang

2021

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…D-S evidence theory has been widely used in information fusion, uncertain reasoning, and other fields. It has a solid mathematical foundation and obtains good fusion results by a simple reasoning form without prior probability [36,37]. Despite these achievements, human-posture recognition remains insufficient for industrial HRC systems.…”

Section: Related Workmentioning

confidence: 99%

Standing-Posture Recognition in Human–Robot Collaboration Based on Deep Learning and the Dempster–Shafer Evidence Theory

Guan

Liu

Cai

et al. 2020

Sensors

View full text Add to dashboard Cite

During human–robot collaborations (HRC), robot systems must accurately perceive the actions and intentions of humans. The present study proposes the classification of standing postures from standing-pressure images, by which a robot system can predict the intended actions of human workers in an HRC environment. To this end, it explores deep learning based on standing-posture recognition and a multi-recognition algorithm fusion method for HRC. To acquire the pressure-distribution data, ten experimental participants stood on a pressure-sensing floor embedded with thin-film pressure sensors. The pressure data of nine standing postures were obtained from each participant. The human standing postures were discriminated by seven classification algorithms. The results of the best three algorithms were fused using the Dempster–Shafer evidence theory to improve the accuracy and robustness. In a cross-validation test, the best method achieved an average accuracy of 99.96%. The convolutional neural network classifier and data-fusion algorithm can feasibly classify the standing postures of human workers.

show abstract

“…These two types of features are highly complementary. 35 The popular methods 20,22 for pose estimation usually describe human joints as a tree-structured graph and use the dynamic programming algorithm to deduce the positions of every joint. Considering that the framework mentioned in Ref.…”

Section: Pose-based Feature Extractionmentioning

confidence: 99%

“…Since trajectory features and pose features are independent of each other, the score-level fusion 35 is chosen to achieve their integration. Let the final decision matrix be Z f ¼ M tr þ M po , the prediction with the highest score of each row in Z f is selected as the classification result.…”

Section: Features Fusion and Classificationmentioning

confidence: 99%

“…Iqbal et al 27 presented a pictorial structure model to incorporate high-level activity information and then combined the pose-based action recognition with FV encoding of iDT using late fusion. Zhang et al 35 applied an improved score-level fusion to trajectory features and pose features based on the bag-of-visual-words (BoVW) 16 model and Dempster-Shafer evidence theory and demonstrated that score-level fusion is the most effective strategy for the combination of these two types of features. For the trajectory features, inspired by the breakthrough in saliency detection, 36,37 we propose a foreground trajectory extraction method according to the characteristics of video frames.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

et al. 2018

Self Cite

View full text Add to dashboard Cite

Action recognition in realistic scenes is a challenging task in the field of computer vision. Although trajectory-based methods have demonstrated promising performance, background trajectories cannot be filtered out effectively, which leads to a reduction in the ratio of valid trajectories. To address this issue, we propose a saliency-based sampling strategy named foreground trajectories on multiscale hybrid masks (HM-FTs). First, the motion boundary images of each frame are calculated to derive the initial masks. According to the characteristics of action videos, image priors and the synchronous updating mechanism based on cellular automata are exploited to generate an optimized weak saliency map, which will be integrated with a strong saliency map obtained via the multiple kernels boosting algorithm. Then, multiscale hybrid masks are achieved through the collaborative optimization strategy and masks intersection. The compensation schemes are designed to extract a set of foreground trajectories that are closely related to human actions. Finally, a hybrid fusion framework for combining trajectory features and pose features is constructed to enhance the recognition performance. The experimental results on two benchmark datasets demonstrate that the proposed method is effective and improves upon most of the state-of-the-art algorithms.

show abstract

Weighted score-level feature fusion based on Dempster–Shafer evidence theory for action recognition

Cited by 5 publications

References 51 publications

Human activity prediction using saliency-aware motion enhancement and weighted LSTM network

Human activity prediction using saliency-aware motion enhancement and weighted LSTM network

Standing-Posture Recognition in Human–Robot Collaboration Based on Deep Learning and the Dempster–Shafer Evidence Theory

Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

Contact Info

Product

Resources

About