Single Shot Temporal Action Detection

Lin, Tianwei; Zhao, Xu; Shou, Zheng

doi:10.1145/3123266.3123343

Cited by 450 publications

(311 citation statements)

References 47 publications

Supporting

Mentioning

296

Contrasting

Unclassified

Order By: Relevance

“…Temporal Action localization has attracted increasing attention in the last several years [6,18,26,33,34]. Inspired by the success of object detection, most current action detection methods resort to the two-stage pipeline: they first generate a set of 1D temporal proposals and then perform classification and temporal boundary regression on each proposal individually.…”

Section: Introductionmentioning

confidence: 99%

Graph Convolutional Networks for Temporal Action Localization

Zeng

Huang

Gan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

522

233

View full text Add to dashboard Cite

Most state-of-the-art action localization systems process each action proposal individually, without explicitly exploiting their relations during learning. However, the relations between proposals actually play an important role in action localization, since a meaningful action always consists of multiple proposals in a video. In this paper, we propose to exploit the proposal-proposal relations using Graph Convolutional Networks (GCNs). First, we construct an action proposal graph, where each proposal is represented as a node and their relations between two proposals as an edge. Here, we use two types of relations, one for capturing the context information for each proposal and the other one for characterizing the correlations between distinct actions. Then we apply the GCNs over the graph to model the relations among different proposals and learn powerful representations for the action classification and localization. Experimental results show that our approach significantly outperforms the state-of-the-art on THUMOS14 (49.1% versus 42.8%). Moreover, augmentation experiments on ActivityNet also verify the efficacy of modeling action proposal relationships.

show abstract

Section: Introductionmentioning

confidence: 99%

Graph Convolutional Networks for Temporal Action Localization

Zeng

Huang

Gan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

522

233

View full text Add to dashboard Cite

show abstract

“…Structured segment network (SSN) [25] proposes to model the temporal structure of each action instance via a structured temporal pyramid. SSAD [26] is proposed to use 1D temporal convolutional layers to skip the proposal generation step via directly detecting action instances in untrimmed videos. There are also some deep networks generating temporal proposals by probability estimation, such as Boundary Sensitive Network [27].…”

Section: B Action Localizationmentioning

confidence: 99%

AdapNet: Adaptability Decomposing Encoder–Decoder Network for Weakly Supervised Action Recognition and Localization

Zhang

Shi

et al. 2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

The point process is a solid framework to model sequential data, such as videos, by exploring the underlying relevance. As a challenging problem for high-level video understanding, weakly supervised action recognition and localization in untrimmed videos has attracted intensive research attention. Knowledge transfer by leveraging the publicly available trimmed videos as external guidance is a promising attempt to make up for the coarse-grained video-level annotation and improve the generalization performance. However, unconstrained knowledge transfer may bring about irrelevant noise and jeopardize the learning model. This paper proposes a novel adaptability decomposing encoder-decoder network to transfer reliable knowledge between trimmed and untrimmed videos for action recognition and localization via bidirectional point process modeling, given only video-level annotations. By decomposing the original features into domain-adaptable and domain-specific ones based on their adaptability, trimmed-untrimmed knowledge transfer can be safely confined within a more coherent subspace. An encoder-decoder based structure is carefully designed and jointly optimized to facilitate effective action classification and temporal localization. Extensive experiments are conducted on two benchmark datasets (i.e., THUMOS14 and ActivityNet1.3), and experimental results clearly corroborate the efficacy of our method.

show abstract

“…We compare the following advanced approaches: (1) Structure Segment Network (SSN) [45] generates action proposals by temporal actionness grouping. (2) Single Shot Action Detection (SSAD) [19] is the 1D variant version of Single Shot Detection [23], which generates action proposals by multiple temporal anchor layers. (3) Convolution-De-Convolution Network (CDC) [31] builds a 3D Conv-Deconv network to precisely localize the boundary of action instances at frame level.…”

Section: Evaluation On Temporal Action Proposalmentioning

confidence: 99%

“…One natural way of temporal action localization is to extend image object detection frameworks, e.g., SSD [23] or Faster R-CNN [29], for producing spatial bounding boxes in a 2D image to temporal localization of an action in a 1D sequence [4,19]. The upper part of Figure 1 conceptualizes a typical process of one-stage action localization.…”

Section: Introductionmentioning

confidence: 99%

Gaussian Temporal Awareness Networks for Action Localization

Long

Yao

Qiu

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

342

149

View full text Add to dashboard Cite

Temporally localizing actions in a video is a fundamental challenge in video understanding. Most existing approaches have often drawn inspiration from image object detection and extended the advances, e.g., SSD and Faster R-CNN, to produce temporal locations of an action in a 1D sequence. Nevertheless, the results can suffer from robustness problem due to the design of predetermined temporal scales, which overlooks the temporal structure of an action and limits the utility on detecting actions with complex variations. In this paper, we propose to address the problem by introducing Gaussian kernels to dynamically optimize temporal scale of each action proposal. Specifically, we present Gaussian Temporal Awareness Networks (GTAN)a new architecture that novelly integrates the exploitation of temporal structure into an one-stage action localization framework. Technically, GTAN models the temporal structure through learning a set of Gaussian kernels, each for a cell in the feature maps. Each Gaussian kernel corresponds to a particular interval of an action proposal and a mixture of Gaussian kernels could further characterize action proposals with various length. Moreover, the values in each Gaussian curve reflect the contextual contributions to the localization of an action proposal. Extensive experiments are conducted on both THUMOS14 and ActivityNet v1.3 datasets, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, GTAN achieves 1.9% and 1.1% improvements in mAP on testing set of the two datasets.

show abstract

Single Shot Temporal Action Detection

Cited by 450 publications

References 47 publications

Graph Convolutional Networks for Temporal Action Localization

Graph Convolutional Networks for Temporal Action Localization

AdapNet: Adaptability Decomposing Encoder–Decoder Network for Weakly Supervised Action Recognition and Localization

Gaussian Temporal Awareness Networks for Action Localization

Contact Info

Product

Resources

About