Action recognition by saliency-based dense sampling

Xu, Zengmin; Hu, Ruimin; Chen, Chen; Chen, Huafeng; Li, Hongyang; Sun, Quan

doi:10.1016/j.neucom.2016.09.106

Cited by 20 publications

(19 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Table 1 Comparison of results on UFC50. accuracy Wang+2013 (DT) [14] 83.6 Kataoka+2015 [15] 84.5 Beaudry+2016 [16] 88.3 TS skip2 (ours) 89.4 Li+2016 [17] 90.3 Wang&Schmid 2013 (iDT) [6] 91.7 Peng+2016 [18] 92.3 Yang+2017 [19] 92.4 Lan+2015 [20] 93.8 Lan+2015 [21] 94.4 Xu+2017 [22] 94.8 TS (ours) 95.0 Duta+2016 [23] 97 .8 reason is that many categories in UCF101 have videos of almost static scenes. For example, in "playing piano" category, a person plays a piano indoor and only small portion of the hands move very small amount.…”

Section: Resultsmentioning

confidence: 99%

Trajectory-Set Feature for Action Recognition

Matsui

Tamaki

Raytchev

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYWe propose a feature for action recognition called Trajectory-Set (TS), on top of the improved Dense Trajectory (iDT). The TS feature encodes only trajectories around densely sampled interest points, without any appearance features. Experimental results on the UCF50 action dataset demonstrates that TS is comparable to state-of-thearts, and outperforms iDT; the accuracy of 95.0%, compared to 91.7% by iDT.

show abstract

Section: Resultsmentioning

confidence: 99%

Trajectory-Set Feature for Action Recognition

Matsui

Tamaki

Raytchev

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…In Table XII, the background modeling method is compared with Mean filter [43] and ViBe [44]. We claim that our method 3 Dataset provided in http://pan.baidu.com/s/1gf9ZmCz 4 Code provided in http://www.cs.ucf.edu/ ∼ oreifej/HON4D.html 5 Code provided in http://cvrc.ece.utexas.edu/lu/index.html 1 29 0 Table XIII, we compare the testing speed of our method with those of HON4D [19] and Depth Gradients+RDF [63]. We test all testing sequences and report the average time cost for each frame.…”

Section: G Computational Costmentioning

confidence: 96%

“…However, too many scales of 3DLSK may contain redundant information and will decrease the discriminative power, which explains the poor performance when L is equal to 5. For each dataset, we select the proper value of the scale by grid search for the grid [1,2,3,4,5], where a five-fold cross-validation method is used on the training data. Table IV evaluates the complementarity property between M3DLSK and STV.…”

Section: Msr-gesture3dmentioning

confidence: 99%

“…Determining how to recognize actions (e.g., hugging, hand waving, smoking) accurately in a cost-effective manner remains one of the main challenges for applications such as human-computer interaction (HCI), content-based video analysis and intelligent surveillance [1], [2], [3], [4]. Previous methods [5], [6], [7], [8] have used conventional color cameras to record actions as RGB sequences and developed distinctive action representations to improve the recognition accuracy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robust 3D Action Recognition Through Sampling Local Appearances and Global Distributions

Liu

Chen

2018

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

3D action recognition has broad applications in human-computer interaction and intelligent surveillance. However, recognizing similar actions remains challenging since previous literature fails to capture motion and shape cues effectively from noisy depth data. In this paper, we propose a novel two-layer Bag-of-Visual-Words (BoVW) model, which suppresses the noise disturbances and jointly encodes both motion and shape cues. First, background clutter is removed by a background modeling method that is designed for depth data. Then, motion and shape cues are jointly used to generate robust and distinctive spatialtemporal interest points (STIPs): motion-based STIPs and shapebased STIPs. In the first layer of our model, a multi-scale 3D local steering kernel (M3DLSK) descriptor is proposed to describe local appearances of cuboids around motion-based STIPs. In the second layer, a spatial-temporal vector (STV) descriptor is proposed to describe the spatial-temporal distributions of shapebased STIPs. Using the BoVW model, motion and shape cues are combined to form a fused action representation. Our model performs favorably compared with common STIP detection and description methods. Thorough experiments verify that our model is effective in distinguishing similar actions and robust to background clutter, partial occlusions and pepper noise.Index Terms-depth data, human-computer interaction (HCI), spatial-temporal interest point (STIP), 3D action recognition.

show abstract

“…However, its recognition accuracy has not improved compared to DT. iDT-RCB 50 is an improved strategy based on DT, where the warped optical flow is exploited to adjust the interest points sampling to remove subtle motions. Although the recognition accuracy of DT is enhanced, its computational cost is higher than DT, which should be attributed to the calculation of optical flow and saliency detection.…”

Section: Comparison Of the Performance Of Different Trajectory Featuresmentioning

confidence: 99%

Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

et al. 2018

View full text Add to dashboard Cite

Action recognition in realistic scenes is a challenging task in the field of computer vision. Although trajectory-based methods have demonstrated promising performance, background trajectories cannot be filtered out effectively, which leads to a reduction in the ratio of valid trajectories. To address this issue, we propose a saliency-based sampling strategy named foreground trajectories on multiscale hybrid masks (HM-FTs). First, the motion boundary images of each frame are calculated to derive the initial masks. According to the characteristics of action videos, image priors and the synchronous updating mechanism based on cellular automata are exploited to generate an optimized weak saliency map, which will be integrated with a strong saliency map obtained via the multiple kernels boosting algorithm. Then, multiscale hybrid masks are achieved through the collaborative optimization strategy and masks intersection. The compensation schemes are designed to extract a set of foreground trajectories that are closely related to human actions. Finally, a hybrid fusion framework for combining trajectory features and pose features is constructed to enhance the recognition performance. The experimental results on two benchmark datasets demonstrate that the proposed method is effective and improves upon most of the state-of-the-art algorithms.

show abstract

Action recognition by saliency-based dense sampling

Cited by 20 publications

References 65 publications

Trajectory-Set Feature for Action Recognition

Trajectory-Set Feature for Action Recognition

Robust 3D Action Recognition Through Sampling Local Appearances and Global Distributions

Saliency-based foreground trajectory extraction using multiscale hybrid masks for action recognition

Contact Info

Product

Resources

About