CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos

Shou, Zheng; Chan, Jack; Zareian, Alireza; Miyazawa, Keiji; Chang, Shih‐Fu

doi:10.1109/cvpr.2017.155

Cited by 578 publications

(396 citation statements)

References 68 publications

Supporting

Mentioning

395

Contrasting

Order By: Relevance

“…Table 2 reports the action localization results of various methods. Regarding the average mAP, P-GCN outperforms SSN [52], CDC [33], and TAL-Net [6] by 3.01%, 3.19%, and 6.77%, respectively. We observe that the method by Lin et al [27] (called LIN below) performs promisingly on this dataset.…”

Section: Comparison With State-of-the-art Resultsmentioning

confidence: 98%

“…Temporal Action localization has attracted increasing attention in the last several years [6,18,26,33,34]. Inspired by the success of object detection, most current action detection methods resort to the two-stage pipeline: they first generate a set of 1D temporal proposals and then perform classification and temporal boundary regression on each proposal individually.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Graph Convolutional Networks for Temporal Action Localization

Zeng

Huang

Gan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

516

233

View full text Add to dashboard Cite

Most state-of-the-art action localization systems process each action proposal individually, without explicitly exploiting their relations during learning. However, the relations between proposals actually play an important role in action localization, since a meaningful action always consists of multiple proposals in a video. In this paper, we propose to exploit the proposal-proposal relations using Graph Convolutional Networks (GCNs). First, we construct an action proposal graph, where each proposal is represented as a node and their relations between two proposals as an edge. Here, we use two types of relations, one for capturing the context information for each proposal and the other one for characterizing the correlations between distinct actions. Then we apply the GCNs over the graph to model the relations among different proposals and learn powerful representations for the action classification and localization. Experimental results show that our approach significantly outperforms the state-of-the-art on THUMOS14 (49.1% versus 42.8%). Moreover, augmentation experiments on ActivityNet also verify the efficacy of modeling action proposal relationships.

show abstract

Section: Comparison With State-of-the-art Resultsmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Graph Convolutional Networks for Temporal Action Localization

Zeng

Huang

Gan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

516

233

View full text Add to dashboard Cite

show abstract

“…In the future, we would consider integrating the early action prediction with action localization [44], [56] together to form a more complete early action analysis system.…”

Section: Discussionmentioning

confidence: 99%

Early Action Prediction by Soft Regression

Zheng

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

101

View full text Add to dashboard Cite

Abstract-We propose a novel approach for predicting on-going action with the assistance of a low-cost depth camera. Our approach introduces a soft regression-based early prediction framework. In this framework, we estimate soft labels for the subsequences at different progress levels, jointly learned with an action predictor. Our formulation of soft regression framework 1) overcomes a usual assumption in existing early action prediction systems that the progress level of on-going sequence is given in the testing stage; and 2) presents a theoretical framework to better resolve the ambiguity and uncertainty of subsequences at early performing stage. The proposed soft regression framework is further enhanced in order to take the relationships among subsequences and the discrepancy of soft labels over different classes into consideration, so that a Multiple Soft labels Recurrent Neural Network (MSRNN) is finally developed. For real-time performance, we also introduce a new RGB-D feature called "local accumulative frame feature (LAFF)", which can be computed efficiently by constructing an integral feature map. Our experiments on three RGB-D benchmark datasets and an unconstrained RGB action set demonstrate that the proposed regression-based early action prediction model outperforms existing models significantly and also show that the early action prediction on RGB-D sequence is more accurate than that on RGB channel.

show abstract

“…Once the localization of action proposals completes, the natural way for temporal action detection is to further classify the proposals into known action classes, making the process in two-stage manner [4,12,31,32,40,45]. However, the separate of proposal generation and classification may result in sub-optimal solutions.…”

Section: Related Workmentioning

confidence: 99%

Gaussian Temporal Awareness Networks for Action Localization

Long

Yao

Qiu

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

340

149

View full text Add to dashboard Cite

Temporally localizing actions in a video is a fundamental challenge in video understanding. Most existing approaches have often drawn inspiration from image object detection and extended the advances, e.g., SSD and Faster R-CNN, to produce temporal locations of an action in a 1D sequence. Nevertheless, the results can suffer from robustness problem due to the design of predetermined temporal scales, which overlooks the temporal structure of an action and limits the utility on detecting actions with complex variations. In this paper, we propose to address the problem by introducing Gaussian kernels to dynamically optimize temporal scale of each action proposal. Specifically, we present Gaussian Temporal Awareness Networks (GTAN)a new architecture that novelly integrates the exploitation of temporal structure into an one-stage action localization framework. Technically, GTAN models the temporal structure through learning a set of Gaussian kernels, each for a cell in the feature maps. Each Gaussian kernel corresponds to a particular interval of an action proposal and a mixture of Gaussian kernels could further characterize action proposals with various length. Moreover, the values in each Gaussian curve reflect the contextual contributions to the localization of an action proposal. Extensive experiments are conducted on both THUMOS14 and ActivityNet v1.3 datasets, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, GTAN achieves 1.9% and 1.1% improvements in mAP on testing set of the two datasets.

show abstract

CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos

Cited by 578 publications

References 68 publications

Graph Convolutional Networks for Temporal Action Localization

Graph Convolutional Networks for Temporal Action Localization

Early Action Prediction by Soft Regression

Gaussian Temporal Awareness Networks for Action Localization

Contact Info

Product

Resources

About