Generalized zero-shot learning for action recognition with web-scale video data

Liu, Kun; Liu, Wu; Ma, Huadóng; Huang, Wenbing; Dong, Xiongxiong

doi:10.1007/s11280-018-0642-6

Cited by 38 publications

(18 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recognizing anomaly in surveillance video [40], [41] can stop illegal activities and guarantee public safety. For example, Algorithm 1 Compress the model with Huffman Coding.…”

Section: A Application I: Anomaly Recognition In Surveillance Videosmentioning

confidence: 99%

A Real-Time Action Representation With Temporal Encoding and Deep Compression

Liu

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

Deep neural networks have achieved remarkable success for video-based action recognition. However, most of existing approaches cannot be deployed in practice due to the high computational cost. To address this challenge, we propose a new real-time convolutional architecture, called Temporal Convolutional 3D Network (T-C3D), for action representation. T-C3D learns video action representations in a hierarchical multi-granularity manner while obtaining a high process speed. Specifically, we propose a residual 3D Convolutional Neural Network (CNN) to capture complementary information on the appearance of a single frame and the motion between consecutive frames. Based on this CNN, we develop a new temporal encoding method to explore the temporal dynamics of the whole video. Furthermore, we integrate deep compression techniques with T-C3D to further accelerate the deployment of models via reducing the size of the model. By these means, heavy calculations can be avoided when doing the inference, which enables the method to deal with videos beyond real-time speed while keeping promising performance. Our method achieves clear improvements on UCF101 action recognition benchmark against state-of-the-art real-time methods by 5.4% in terms of accuracy and 2 times faster in terms of inference speed with a less than 5MB storage model. We validate our approach by studying its action representation performance on four different benchmarks over three different tasks. Extensive experiments demonstrate comparable recognition performance to the stateof-the-art methods. The source code and the pre-trained models are publicly available at https://github.com/tc3d.

show abstract

“…Recognizing anomaly in surveillance video [40], [41] can stop illegal activities and guarantee public safety. For example, Algorithm 1 Compress the model with Huffman Coding.…”

Section: A Application I: Anomaly Recognition In Surveillance Videosmentioning

confidence: 99%

A Real-Time Action Representation With Temporal Encoding and Deep Compression

Liu

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…All the aforementioned strategies are possible solutions to an FSL problem. FSL has already been introduced as a means of dealing with emergency situations [30][31][32]. However, the related studies do not address post-disaster emergency mapping explicitly but rather focus on video surveillance [32], tweet classification [30] or indoors safety [31].…”

Section: Study Data Source Dataset Size ML Approach Classesmentioning

confidence: 99%

Few-Shot Learning for Post-Earthquake Urban Damage Detection

2021

View full text Add to dashboard Cite

Among natural disasters, earthquakes are recorded to have the highest rates of human loss in the past 20 years. Their unexpected nature has severe consequences on both human lives and material infrastructure, demanding urgent action to be taken. For effective emergency relief, it is necessary to gain awareness about the level of damage in the affected areas. The use of remotely sensed imagery is popular in damage assessment applications; however, it requires a considerable amount of labeled data, which are not always easy to obtain. Taking into consideration the recent developments in the fields of Machine Learning and Computer Vision, this study investigates and employs several Few-Shot Learning (FSL) strategies in order to address data insufficiency and imbalance in post-earthquake urban damage classification. While small datasets have been tested against binary classification problems, which usually divide the urban structures into collapsed and non-collapsed, the potential of limited training data in multi-class classification has not been fully explored. To tackle this gap, four models were created, following different data balancing methods, namely cost-sensitive learning, oversampling, undersampling and Prototypical Networks. After a quantitative comparison among them, the best performing model was found to be the one based on Prototypical Networks, and it was used for the creation of damage assessment maps. The contribution of this work is twofold: we show that oversampling is the most suitable data balancing method for training Deep Convolutional Neural Networks (CNN) when compared to cost-sensitive learning and undersampling, and we demonstrate the appropriateness of Prototypical Networks in the damage classification context.

show abstract

“…For testing, only temporal labels are available, meaning spatial evaluation cannot be done. While this General video surveillance/recognition datasets such as [19,41,26,27] have not been used to evaluate video anomaly detection since they are not specifically curated for this purpose and do not contain sufficient ground truth annotations.…”

Section: Ucsd Pedestrianmentioning

confidence: 99%

Street Scene: A new dataset and evaluation protocol for video anomaly detection

Ramachandra¹,

Jones²

2019

Preprint

View full text Add to dashboard Cite

Progress in video anomaly detection research is currently slowed by small datasets that lack a wide variety of activities as well as flawed evaluation criteria. This paper aims to help move this research effort forward by introducing a large and varied new dataset called Street Scene, as well as two new evaluation criteria that provide a better estimate of how an algorithm will perform in practice. In addition to the new dataset and evaluation criteria, we present two variations of a novel baseline video anomaly detection algorithm and show they are much more accurate on Street Scene than two well known algorithms from the literature.

show abstract

Generalized zero-shot learning for action recognition with web-scale video data

Cited by 38 publications

References 54 publications

A Real-Time Action Representation With Temporal Encoding and Deep Compression

A Real-Time Action Representation With Temporal Encoding and Deep Compression

Few-Shot Learning for Post-Earthquake Urban Damage Detection

Street Scene: A new dataset and evaluation protocol for video anomaly detection

Contact Info

Product

Resources

About