Hybrid Active Learning via Deep Clustering for Video Action Detection

Rana, Aayush J; Rawat, Yogesh S

doi:10.1109/cvpr52729.2023.01809

Cited by 10 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Implementation details: We use the PyTorch to build our models and train them on single 16GB GPU. For action detection, we use VideoCapsuleNet (Duarte, Rawat, and Shah 2018;Kumar and Rawat 2022;Rana and Rawat 2022), with margin-loss for classification and BCE loss for detection. The network input is 8 RGB frames of size 224 × 224.…”

Section: Methodsmentioning

confidence: 99%

“…Active learning (Pardo et al 2021) enables selecting samples for annotation by estimating the usefulness of each sample to the underlying task. It is used to iteratively select a subset of data for annotation on various tasks as image classification (Wang et al 2016), image object detection (Aghdam et al 2019;Pardo et al 2021) and video temporal localization (Heilbron et al 2018) with only few studies done for video action detection (Rana and Rawat 2022). The sample selection in AL is done using uncertainty (Liu et al 2019), entropy (Aghdam et al 2019), core-set selection (Sener and Savarese 2017) or mutual-information (Kirsch, Van Amersfoort, and Gal 2019).…”

Section: Related Workmentioning

confidence: 99%

“…Weakly supervised methods often underperform compared to super-vised methods, often requiring externally trained object detector to address the detection aspect of action detection (Chéron et al 2018;Weinzaepfel, Martin, and Schmid 2016). Recently, semi-supervised learning (SSL) (Kumar and Rawat 2022) and active learning methods (Rana and Rawat 2022) have shown promising performance. However, they also have their own limitations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semi-supervised Active Learning for Video Action Detection

Singh,

Rana,

Kumar

et al. 2024

AAAI

View full text Add to dashboard Cite

In this work, we focus on label efficient learning for video action detection. We develop a novel semi-supervised active learning approach which utilizes both labeled as well as un- labeled data along with informative sample selection for ac- tion detection. Video action detection requires spatio-temporal localization along with classification, which poses several challenges for both active learning (informative sample se- lection) as well as semi-supervised learning (pseudo label generation). First, we propose NoiseAug, a simple augmenta- tion strategy which effectively selects informative samples for video action detection. Next, we propose fft-attention, a novel technique based on high-pass filtering which enables effective utilization of pseudo label for SSL in video action detection by emphasizing on relevant activity region within a video. We evaluate the proposed approach on three different bench- mark datasets, UCF-101-24, JHMDB-21, and Youtube-VOS. First, we demonstrate its effectiveness on video action detec- tion where the proposed approach outperforms prior works in semi-supervised and weakly-supervised learning along with several baseline approaches in both UCF101-24 and JHMDB- 21. Next, we also show its effectiveness on Youtube-VOS for video object segmentation demonstrating its generalization capability for other dense prediction tasks in videos.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semi-supervised Active Learning for Video Action Detection

Singh,

Rana,

Kumar

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

Two-Stage Active Learning for Efficient Temporal Action Segmentation

Su,

Elhamifar

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

An effective Key Frame Extraction technique based on Feature Fusion and Fuzzy-C means clustering with Artificial Hummingbird

Kaur,

Lal

2024

Sci Rep

View full text Add to dashboard Cite

Key frame extraction is very important in video summarization and content-based video analysis to address the problem of data redundancy in a video. Key frame extraction enables quick navigation and expert video arrangement in many applications. The visually impaired can benefit from the use of key frame extraction for rapid object recognition and tracking. Most key frame extraction techniques consider only a single visual feature instead of multiple features or full pictorial information of the video. This study proposes a key frame extraction method from a video that (i) first removes insignificant frames by pre-processing, (ii) second, four visual and structural feature differences among the consecutive frames are extracted and aggregated to identify informative frames, (iii) third, to cluster the obtained frames, a hybrid FCM-AHA method is proposed by combining Fuzzy C-means(FCM) with artificial hummingbird optimization algorithm (AHA) to circumvent the local minima trapping problem of FCM, and finally, from each cluster, the two frames having greatest Euclidean distance from all the other frames within a cluster is selected as key frames to remove redundant frames. Experimental results on the Open video and YouTube datasets show that the suggested method outperforms state-of-the-art methods both in terms of subjective qualitative analysis and objective quantitative evaluation, e.g., Precision, Recall, and F-score. Further, results are also taken on real video to demonstrate its applicability in real-life applications.

show abstract

Hybrid Active Learning via Deep Clustering for Video Action Detection

Cited by 10 publications

References 39 publications

Semi-supervised Active Learning for Video Action Detection

Semi-supervised Active Learning for Video Action Detection

Two-Stage Active Learning for Efficient Temporal Action Segmentation

An effective Key Frame Extraction technique based on Feature Fusion and Fuzzy-C means clustering with Artificial Hummingbird

Contact Info

Product

Resources

About