Learning to Discriminate Information for Online Action Detection

Eun, Hyunjun; Moon, Jinyoung; Park, Jongyoul; Jung, Chanho; Kim, Changick

doi:10.1109/cvpr42600.2020.00089

Cited by 74 publications

(62 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When combining with Rep. Flow [27], our result reaches 44.4%. RED [15], TRN [16] and IDN [24] indeed report better results by exploiting optical flow features, which are more accurate than those of the fast approximated version, i.e., Rep. Flow. But it should be noted that the computation of standard optical flows is very expensive.…”

Section: Resultsmentioning

confidence: 85%

Human-Aware Coarse-to-Fine Online Action Detection

Yang

Huang

Qin

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this work, we propose a two-stage framework to efficiently and effectively detect actions on-the-fly. An action location network (ALN) is developed in the first stage to judge whether the current frame is action-related, while the second stage involves an action classification network (ACN) to further identify the action category. In this way, irrelevant negative frames are quickly discarded and actions are detected as early as they occur. Moreover, we highlight human areas at both the stages by respectively incorporating a human detector and a human mask layer. As a result, more accurate spatial-temporal windows of actions are detected, based on which more robust features are extracted for classification. Experimental results on two popular benchmarks demonstrate the superior performance of the proposed approach.

show abstract

Section: Resultsmentioning

confidence: 85%

Human-Aware Coarse-to-Fine Online Action Detection

Yang

Huang

Qin

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…By combining LSTM, the long-term temporal dependencies can be further accumulated at the last hidden states, but this operation also lead to the irrelevant information to be accumulated, especially for TVSeries dataset, since different actions can occur at the same time and being performed by the same or multiple actors, as opposed to the setting of THUMOS-14, where actions are separated by a specific non-action. (5) Comparing M3→M4 and M5→M6, the order of LSTM and DCC makes difference on both datasets, the DCCfirst order performs better than the LSTM-first order especially for TVSeries, showing the effectiveness of The top-2 results are boldly marked and [15] is available online after our submission Single-frame CNN [59] Two-stream CNN [58] C3D+LinearInterp [55] Predictive-corrective [9] LSTM [12] MultiLSTM [74] CDC [55] RED [20] TRN [72] IDN [15] 50.0…”

Section: Hybrid Temporal Modeling Methodsmentioning

confidence: 99%

“…The top-2 results are boldly marked and [15] is available online after our submission DCC to discriminate relevant and filter out irrelevant information for online action detection.…”

Section: Hybrid Temporal Modeling Methodsmentioning

confidence: 99%

“…Xu et al [72] proposed the Temporal Recurrent Network (TRN) to model the temporal context by simultaneously performing online action detection and anticipation. Eun et al [15] designed a novel recurrent unit named Information Discrimination Unit to explicitly discriminate the information relevant to an ongoing action from others. Besides, Shou et al [56] formulated the online detection of action start (ODAS) as a classification task of sliding windows and introduced a model based on Generative Adversarial Network (GAN) to generate hard negative samples to improve the training of the samples.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An empirical study on temporal modeling for online action detection

Wang

Peng

Qiao

et al. 2021

Complex Intell. Syst.

View full text Add to dashboard Cite

Online action detection (OAD) is a practical yet challenging task, which has attracted increasing attention in recent years. A typical OAD system mainly consists of three modules: a frame-level feature extractor which is usually based on pre-trained deep Convolutional Neural Networks (CNNs), a temporal modeling module, and an action classifier. Among them, the temporal modeling module is crucial which aggregates discriminative information from historical and current features. Though many temporal modeling methods have been developed for OAD and other topics, their effects are lack of investigation on OAD fairly. This paper aims to provide an empirical study on temporal modeling for OAD including four meta types of temporal modeling methods, i.e. temporal pooling, temporal convolution, recurrent neural networks, and temporal attention, and uncover some good practices to produce a state-of-the-art OAD system. Many of them are explored in OAD for the first time, and extensively evaluated with various hyper parameters. Furthermore, based on our empirical study, we present several hybrid temporal modeling methods. Our best networks, i.e. , the hybridization of DCC, LSTM and M-NL, and the hybridization of DCC and M-NL, which outperform previously published results with sizable margins on THUMOS-14 dataset (48.6% vs. 47.2%) and TVSeries dataset (84.3% vs. 83.7%).

show abstract

“…Human posture estimation is one of the important applications of deep convolution neural network in behavior perception [1][2][3]. It has been widely utilized in pedestrian movement trend detection, health recovery training and other related fields, working by capturing image signals of human behaviors, tracking and judging different behavior postures [4][5][6].…”

Section: Introductionmentioning

confidence: 99%

Multi-Cascade Perceptual Human Posture Recognition Enhancement Network

et al. 2021

IEEE Access

View full text Add to dashboard Cite

The current researches trend to adopt a low-resolution hot spot map to restore the original highresolution representation to save computing cost, resulting in unsatisfactory detection performance, especially in human body recognition with a highly complex background. Aiming at this problem, we proposed a model of parallel connection of multiple sub-networks with different resolution levels on a high-resolution main network. It can maintain the network structure of a high-resolution hot spot map in the whole operation process. By using this structure in the human key point vector field network, the accuracy of human posture recognition has been improved with high-speed operation. To validate the proposed model's effectiveness, two common benchmark data sets of COCO key point data set and MPII human posture data set are used for evaluation. Experimental results show that our network achieves the accuracy of 72.3% AP and 92.2% AP in the two data sets, respectively, which is 3%-4% higher than those of the existing mainstream researches. In our test, only the accuracy of backbone's SimpleBaseline with ResNet-152 is close to ours, yet our network realized a much lower computing cost.

show abstract

Learning to Discriminate Information for Online Action Detection

Cited by 74 publications

References 24 publications

Human-Aware Coarse-to-Fine Online Action Detection

Human-Aware Coarse-to-Fine Online Action Detection

An empirical study on temporal modeling for online action detection

Multi-Cascade Perceptual Human Posture Recognition Enhancement Network

Contact Info

Product

Resources

About