Weakly supervised activity analysis with spatio-temporal localisation

Gu, Feng; Sridhar, Muralikrishna; Cohn, Anthony G.; Hogg, David C.; Flrez-Revuelta, Francisco; Monekosso, Dorothy; Remagnino, Paolo

doi:10.1016/j.neucom.2016.08.032

Cited by 5 publications

(2 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, some recent works tend to explore object-based representations without using any explicit semantic object segmentation techniques, which seeks a synergy between the MIML and object-level representations. [11] address this weakly supervised issue in multi-label human action detection with a two-stage solution. First, a set of potential objects or spatial-temporal volumes are generated and selected from a video instance with a set of handcrafted rules.…”

Section: Multi-label Learningmentioning

confidence: 99%

See 1 more Smart Citation

Multi-label zero-shot human action recognition via joint latent ranking embedding

Wang

Chen

2020

Neural Networks

View full text Add to dashboard Cite

Human action recognition refers to automatic recognizing human actions from a video clip, which is one of the most challenging tasks in computer vision. Due to the fact that annotating video data is laborious and time-consuming, most of the existing works in human action recognition are limited to a number of small scale benchmark datasets where there are a small number of video clips associated with only a few human actions and a video clip often contains only a single action. In reality, however, there often exist multiple human actions in a video stream. Such a video stream is often weakly-annotated with a set of relevant human action labels at a global level rather than assigning each label to a specific video episode corresponding to a single action, which leads to a multi-label learning problem. Furthermore, there are a great number of meaningful human actions in reality but it would be extremely difficult, if not impossible, to collect/annotate video clips regarding all of various human actions, which leads to a zero-shot learning scenario. To the best of our knowledge, there is no work that has addressed all the above issues together in human action recognition. In this paper, we formulate a real-world human action recognition task as a multi-label zero-shot learning problem and propose a framework to tackle this problem in a holistic way. Our framework holistically tackles the issue of unknown temporal boundaries between different actions for multi-label learning and exploits the side information regarding the semantic relationship between different human actions for knowledge transfer. As a result, our framework leads to a joint latent ranking embedding for multi-label zero-shot human action recognition. A novel neural architecture of two component models and an alternate learning algorithm are proposed to carry out the joint latent ranking embedding learning. Thus, multi-label zero-shot recognition is done by measuring relatedness scores of action labels to a test video clip in the joint latent visual and semantic embedding spaces. We evaluate our framework with different settings, including a novel data split scheme designed especially for evaluating multi-label zero-shot learning, on two weakly annotated multi-label human action datasets: Breakfast and Charades. The experimental results demonstrate the effectiveness of our framework in multi-label zero-shot human action recognition.

show abstract

Section: Multi-label Learningmentioning

confidence: 99%

“…( 8) and ( 9) or the regularized hinge rank loss functions in Eqs. (10) and (11) to train visual and semantic embedding models in our framework.…”

Section: Rank Loss Functionmentioning

confidence: 99%

Multi-label zero-shot human action recognition via joint latent ranking embedding

Wang

Chen

2020

Neural Networks

View full text Add to dashboard Cite

show abstract

Deep Learning for Action and Gesture Recognition in Image Sequences: A Survey

Asadi-Aghbolaghi

Clapés

Bellantonio

et al. 2017

Gesture Recognition

View full text Add to dashboard Cite

Interest in automatic action and gesture recognition has grown considerably in the last few years. This is due in part to the large number of application domains for this type of technology. As in many other computer vision areas, deep learning based methods have quickly become a reference methodology for obtaining state-of-the-art performance in both tasks. This chapter is a survey of current deep learning based methodologies for action and gesture recognition in sequences of images. The survey reviews both fundamental and cutting edge methodologies reported in the last few years. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. Details of the proposed architectures, fusion strategies, main datasets, and competitions are reviewed. *. A reduced version of this appeared appeared as: M. Asadi-Aghbolaghi et al. A survey on deep learning based approaches for action and gesture recognition in image sequences. In

show abstract

A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences

Asadi-Aghbolaghi

Clapés

Bellantonio

et al. 2017

2017 12th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2017)

169

View full text Add to dashboard Cite

The interest in action and gesture recognition has grown considerably in the last years. In this paper, we present a survey on current deep learning methodologies for action and gesture recognition in image sequences. We introduce a taxonomy that summarizes important aspects of deep learning for approaching both tasks. We review the details of the proposed architectures, fusion strategies, main datasets, and competitions. We summarize and discuss the main works proposed so far with particular interest on how they treat the temporal dimension of data, discussing their main features and identify opportunities and challenges for future research.

show abstract

Weakly supervised activity analysis with spatio-temporal localisation

Cited by 5 publications

References 41 publications

Multi-label zero-shot human action recognition via joint latent ranking embedding

Multi-label zero-shot human action recognition via joint latent ranking embedding

Deep Learning for Action and Gesture Recognition in Image Sequences: A Survey

A Survey on Deep Learning Based Approaches for Action and Gesture Recognition in Image Sequences

Contact Info

Product

Resources

About