Still Image Action Recognition by Predicting Spatial-Temporal Pixel Evolution

Safaei, Marjaneh; Foroosh, Hassan

doi:10.1109/wacv.2019.00019

Cited by 19 publications

(9 citation statements)

References 103 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Action Recognition from a Single Image. One direction for action recognition is purely based on a single image [10,56,40,5]. In [10], multiple small objects are first identified in a still image and then the target action is inferred from the relationship among the objects.…”

Section: Related Workmentioning

confidence: 99%

Can An Image Classifier Suffice For Action Recognition?

Fan¹,

Chun-Fu²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a new perspective on video understanding by casting the video recognition problem as an image recognition task. We show that an image classifier alone can suffice for video understanding without temporal modeling. Our approach is simple and universal. It composes input frames into a super image to train an image classifier to fulfill the task of action recognition, in exactly the same way as classifying an image. We prove the viability of such an idea by demonstrating strong and promising performance on four public datasets including Kinetics400, Something-to-something (V2), MiT and Jester, using a recently developed vision transformer. We also experiment with the prevalent ResNet image classifiers in computer vision to further validate our idea. The results on Ki-netics400 are comparable to some of the best-performed CNN approaches based on spatio-temporal modeling. our code and models will be made available at https://github.com/IBM/sifar-pytorch.

show abstract

Section: Related Workmentioning

confidence: 99%

Can An Image Classifier Suffice For Action Recognition?

Fan¹,

Chun-Fu²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Khan et al [26] proposed an action-specific person detection based on transfer learning to improve the quality of bounding boxes. Recently, deep learning has produced successful results in single-image-based action recognition [26][27][28][29][30]. Zhang et al [28] proposed action recognition with only action labels, which learns the action masks and extracts the features from the objects in an action.…”

Section: Single-image-based Action Recognitionmentioning

confidence: 99%

Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition

2020

View full text Add to dashboard Cite

Action recognition is an application that, ideally, requires real-time results. We focus on single-image-based action recognition instead of video-based because of improved speed and lower cost of computation. However, a single image contains limited information, which makes single-image-based action recognition a difficult problem. To get an accurate representation of action classes, we propose three feature-stream-based shallow sub-networks (image-based, attention-image-based, and part-image-based feature networks) on the deep pose estimation network in a multitasking manner. Moreover, we design the multitask-aware loss function, so that the proposed method can be adaptively trained with heterogeneous datasets where only human pose annotations or action labels are included (instead of both pose and action information), which makes it easier to apply the proposed approach to new data on behavioral analysis on intelligent systems. In our extensive experiments, we showed that these streams represent complementary information and, hence, the fused representation is robust in distinguishing diverse fine-grained action classes. Unlike other methods, the human pose information was trained using heterogeneous datasets in a multitasking manner; nevertheless, it achieved 91.91% mean average precision on the Stanford 40 Actions Dataset. Moreover, we demonstrated the proposed method can be flexibly applied to multi-labels action recognition problem on the V-COCO Dataset.

show abstract

“…Still image action recognition has recently benefited from the outstanding performance of CNN models (Gao, Xiong, and Grauman 2018;Gkioxari, Girshick, and Malik 2015;Safaei and Foroosh 2019;Hoai 2014;Oquab et al 2014;Rahman and Wang 2016). The tradeoff is the need for millions of parameters and dependency on huge training sets.…”

Section: Related Workmentioning

confidence: 99%

UCF-STAR: A Large Scale Still Image Dataset for Understanding Human Actions

Safaei¹,

Balouchian²,

Foroosh³

2020

AAAI

Self Cite

View full text Add to dashboard Cite

Action recognition in still images poses a great challenge due to (i) fewer available training data, (ii) absence of temporal information. To address the first challenge, we introduce a dataset for STill image Action Recognition (STAR), containing over $1M$ images across 50 different human body-motion action categories. UCF-STAR is the largest dataset in the literature for action recognition in still images. The key characteristics of UCF-STAR include (1) focusing on human body-motion rather than relatively static human-object interaction categories, (2) collecting images from the wild to benefit from a varied set of action representations, (3) appending multiple human-annotated labels per image rather than just the action label, and (4) inclusion of rich, structured and multi-modal set of metadata for each image. This departs from existing datasets, which typically provide single annotation in a smaller number of images and categories, with no metadata. UCF-STAR exposes the intrinsic difficulty of action recognition through its realistic scene and action complexity. To benchmark and demonstrate the benefits of UCF-STAR as a large-scale dataset, and to show the role of “latent” motion information in recognizing human actions in still images, we present a novel approach relying on predicting temporal information, yielding higher accuracy on 5 widely-used datasets.

show abstract

Still Image Action Recognition by Predicting Spatial-Temporal Pixel Evolution

Cited by 19 publications

References 103 publications

Can An Image Classifier Suffice For Action Recognition?

Can An Image Classifier Suffice For Action Recognition?

Body-Part-Aware and Multitask-Aware Single-Image-Based Action Recognition

UCF-STAR: A Large Scale Still Image Dataset for Understanding Human Actions

Contact Info

Product

Resources

About