UCF-STAR: A Large Scale Still Image Dataset for Understanding Human Actions

Abstract

Action recognition in still images poses a great challenge due to (i) fewer available training data, (ii) absence of temporal information. To address the first challenge, we introduce a dataset for STill image Action Recognition (STAR), containing over $1M$ images across 50 different human body-motion action categories. UCF-STAR is the largest dataset in the literature for action recognition in still images. The key characteristics of UCF-STAR include (1) focusing on human body-motion rather than relatively static human-object interaction categories, (2) collecting images from the wild to benefit from a varied set of action representations, (3) appending multiple human-annotated labels per image rather than just the action label, and (4) inclusion of rich, structured and multi-modal set of metadata for each image. This departs from existing datasets, which typically provide single annotation in a smaller number of images and categories, with no metadata. UCF-STAR exposes the intrinsic difficulty of action recognition through its realistic scene and action complexity. To benchmark and demonstrate the benefits of UCF-STAR as a large-scale dataset, and to show the role of “latent” motion information in recognizing human actions in still images, we present a novel approach relying on predicting temporal information, yielding higher accuracy on 5 widely-used datasets.

RCAT: Retentive CLIP Adapter Tuning for Improved Video Recognition

“…To thoroughly evaluate the RCAT framework, we conducted comprehensive comparative and ablation studies, emphasizing key metrics like computational efficiency, inference speed, and model complexity. These evaluations were carried out on prominent video classification datasets: UCF101 [11] and HMDB51 [13]. Moreover, to illustrate the model's robust generalization ability, we performed video retrieval experiments on the MSR-VTT [14] dataset, where our framework achieved state-of-the-art performance.…”

“…This technique meticulously recalibrates the pre-trained CLIP model, enhancing its compatibility with video recognition tasks by fine-tuning the alignment between video and textual features. Consequently, we evolve from Equations ( 8) and (9) to Equations ( 10) and (11):…”

“…UCF101 [11] is an action recognition dataset comprising real-life action videos sourced from YouTube. It offers 13,320 videos from 101 action categories, totaling approximately 27 h. These actions are broadly categorized into five types: human-object interaction, purely body motions, human-human interaction, playing musical instruments, and sports.…”

“…This mechanism proficiently synchronizes video and text features, facilitating a more profound and precise understanding of video content, a pivotal aspect in diverse video recognition scenarios. • Empirical Validation on Benchmark Datasets: We have meticulously evaluated the RCAT model's efficacy using public video recognition datasets, including UCF101 [11] and HMDB51 [12], and the MSR-VTT [13] dataset for video retrieval purposes. Our empirical results reveal notable improvements in recognition precision, highlighting the model's strength and flexibility across different video recognition and retrieval scenarios.…”

Multi-Expert Human Action Recognition with Hierarchical Super-Class Learning

“…In this respect, initial studies in HAR have mostly employed low-level feature extraction techniques to capture low-level structures; however, they fail to achieve reliable and satisfactory results [10]. Another category of approaches explores to leverage object detector [11] or pose estimator [12] developments to detect the keypoint joints [13], which are the most discriminative regions in the foreground area. Such detected areas favorably contribute to the overall action recognition accuracy.…”