A Context Knowledge Map Guided Coarse-to-Fine Action Recognition

Ji, Yanli; Zhan, Yue; Yang, Yang; Xu, Xing; Shen, Fumin; Shen, Heng Tao

doi:10.1109/tip.2019.2952088

Cited by 37 publications

(13 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, they proposed a large-scale video dataset, named Kinetics, to facilitate the development of deep learning-based action recognition. Currently, 3D models are prevalent in state-of-the-art video analysis frameworks [20]- [26] due to its solid performance for spatial-temporal modeling.…”

Section: A Video Recognitionmentioning

confidence: 99%

Action Keypoint Network for Efficient Video Recognition

Chen¹,

Han²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Reducing redundancy is crucial for improving the efficiency of video recognition models. An effective approach is to select informative content from the holistic video, yielding a popular family of dynamic video recognition methods. However, existing dynamic methods focus on either temporal or spatial selection independently while neglecting a reality that the redundancies are usually spatial and temporal, simultaneously. Moreover, their selected content is usually cropped with fixed shapes (e.g., temporally-cropped frames, spatially-cropped patches), while the realistic distribution of informative content can be much more diverse. With these two insights, this paper proposes to integrate temporal and spatial selection into an Action Keypoint Network (AK-Net). From different frames and positions, AK-Net selects some informative points scattered in arbitrary-shaped regions as a set of "action keypoints" and then transforms the video recognition into point cloud classification. More concretely, AK-Net has two steps, i.e., the keypoint selection and the point cloud classification. First, it inputs the video into a baseline network and outputs a feature map from an intermediate layer. We view each pixel on this feature map as a spatialtemporal point and select some informative keypoints using selfattention. Second, AK-Net devises a ranking criterion to arrange the keypoints into an ordered 1D sequence. Since the video is represented with a 1D sequence after the specified layer, AK-Net transforms the subsequent layers into a point cloud classification sub-net by compacting the original 2D convolutional kernels into 1D kernels. Consequentially, AK-Net brings two-fold benefits for efficiency: The keypoint selection step collects informative content within arbitrary shapes and increases the efficiency for modeling spatial-temporal dependencies, while the point cloud classification step further reduces the computational cost by compacting the convolutional kernels. Experimental results show that AK-Net can consistently improve the efficiency and performance of baseline methods on several video recognition benchmarks.

show abstract

Section: A Video Recognitionmentioning

confidence: 99%

Action Keypoint Network for Efficient Video Recognition

Chen¹,

Han²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Action Recognition. Action recognition [4,16,31,33] is a fundamental task in the field of video understanding. The goal of action recognition is to predict the category of a trimmed video.…”

Section: Related Workmentioning

confidence: 99%

Multi-scale Dynamic Network for Temporal Action Detection

Ren

Shen

et al. 2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

In recent years, as the fundamental task in video understanding, Temporal Action Detection is attracting extensive attention. Most existing approaches use the same model parameters to process all input videos, which are not adaptive to the input video during the inference stage. In this paper, we propose a novel model termed Multi-scale Dynamic Network (MDN) to tackle this problem. The proposed MDN model incorporates multiple Multi-scale Dynamic Modules (MDMs). Each MDM can generate video-specific and segment-specific convolution kernels based on video content from different scales and adaptively capture rich semantic information for the prediction. Besides, we also design a new Edge Suppression Loss (ESL) function for MDN to pay more attention to hard examples. Extensive experiments conducted on two popular benchmarks ActivityNet-1.3 and THUMOS-14 show that the proposed MDN model achieves the state-of-the-art performance. CCS CONCEPTS• Computing methodologies → Activity recognition and understanding.

show abstract

“…No matter how good the algorithm is, it needs to be verified by practical application [31], and it should be improved and perfected continuously in this process. So our next step is to explore the possibilities of our algorithm in application scenarios that require EEG analysis.…”

Section: Future Workmentioning

confidence: 99%

“…Furthermore, we would like to apply EEG into a more realistic practical research framework with crossmodal [32,33] and multitask [34] in action recognition [31] and HCI in VR [35].…”

Section: Future Workmentioning

confidence: 99%

Attention Optimization Method for EEG via the TGAM

Xie

2020

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

Since the 21st century, noninvasive brain-computer interface (BCI) has developed rapidly, and brain-computer devices have gradually moved from the laboratory to the mass market. Among them, the TGAM (ThinkGear Asic Module) and its encapsulate algorithm have been adopted by many research teams and faculty members around the world. However, due to the limited development cost, the effectiveness of the algorithm to calculate data is not satisfactory. This paper proposes an attention optimization algorithm based on the TGAM for EEG data feedback. Considering that the data output of the TGAM encapsulate algorithm fluctuates greatly, the delay is high and the accuracy is low. The experimental results demonstrated that our algorithm can optimize EEG data, so that with the same or even lower delay and without changing the encapsulate algorithm of the module itself, it can significantly improve the performance of attention data, greatly improve the stability and accuracy of data, and achieve better results in practical applications.

show abstract

A Context Knowledge Map Guided Coarse-to-Fine Action Recognition

Cited by 37 publications

References 42 publications

Action Keypoint Network for Efficient Video Recognition

Action Keypoint Network for Efficient Video Recognition

Multi-scale Dynamic Network for Temporal Action Detection

Attention Optimization Method for EEG via the TGAM

Contact Info

Product

Resources

About