Background Suppression Network for Weakly-Supervised Temporal Action Localization

Abstract

Weakly-supervised temporal action localization is a very challenging problem because frame-wise labels are not given in the training stage while the only hint is video-level labels: whether each video contains action frames of interest. Previous methods aggregate frame-level class scores to produce video-level prediction and learn from video-level action labels. This formulation does not fully model the problem in that background frames are forced to be misclassified as action classes to predict video-level labels accurately. In this paper, we design Background Suppression Network (BaS-Net) which introduces an auxiliary class for background and has a two-branch weight-sharing architecture with an asymmetrical training strategy. This enables BaS-Net to suppress activations from background frames to improve localization performance. Extensive experiments demonstrate the effectiveness of BaS-Net and its superiority over the state-of-the-art methods on the most popular benchmarks – THUMOS'14 and ActivityNet. Our code and the trained model are available at https://github.com/Pilhyeon/BaSNet-pytorch.

PUNet: Temporal Action Proposal Generation With Positive Unlabeled Learning Using Key Frame Annotations

“…These results on THUMOS'14 are summarized in Table 2. Our method outperforms all weakly supervised methods except BaSNet [17], against which it shows a slight performance decrease while being more data efficient and having a simpler network design. Besides, our iterative approach takes around 4.6 minutes to train even on CPU.…”

A Survey on Temporal Action Localization

“…At the same time, they propose a scheme generating a hard negative video for separating contexts. Although the main point of this article is not the background class, it inspires the next subsequent three works that are BaSNet [36], background modeling [37], and LPAT [38]. Without considering the background category, the background frames were misclassified into action categories, resulting in a large number of FPs.…”

Performing Weakly Supervised Retail Instance Segmentation via Region Normalization

“…First, we select the initial seed for the clustering algorithm. Inspired by researches in information theory [43], [44], we think a feature vector to be more informative if its magnitude is larger. Thus, we calculate the L1 norm for all vectors, sort them and select the one corresponding to the median as the initial seed for the clustering algorithm.…”