Action Classification in Soccer Videos with Long Short-Term Memory Recurrent Neural Networks

Abstract

Abstract. In this paper, we propose a novel approach for action classification in soccer videos using a recurrent neural network scheme. Thereby, we extract from each video action at each timestep a set of features which describe both the visual content (by the mean of a BoW approach) and the dominant motion (with a key point based approach). A Long Short-Term Memory-based Recurrent Neural Network is then trained to classify each video sequence considering the temporal evolution of the features for each timestep. Experimental results on the MICC-Soccer-Actions-4 database show that the proposed approach outperforms classification methods of related works (with a classification rate of 77 %), and that the combination of the two features (BoW and dominant motion) leads to a classification rate of 92 %.

Sequential Deep Learning for Human Action Recognition

“…The second key idea in LSTM is the use of multiplicative gates to control the access to the CEC. We have shown in our previous work [1] that LSTM are efficient to label sequences of descriptors corresponding to hand-crafted features.…”

“…Thus, even if the learned features, taken individually, contains temporal information, their evolution over time is completely ignored. Though, we have shown in our previous work [1] that such information does help discriminating between actions, and is particularly usable by a category of learning machines, adapted to sequential data, namely Long Short-Term Memory recurrent neural networks (LSTM) [6].…”

Predicting Human Interaction via Relative Attention Model

“…Hard attention [Mnih et al, 2014;Ba et al, 2014] samples attention location at each time stamp, which causes the system not differentiable. In contrast, soft attention [Bahdanau et al, 2014;Sharma et al, 2015] aims to learn a set of weights corresponding to each region, the model is differentiable and can be trained end-to-end using standard backpropagation. Therefore, we adopt the soft attention model in our work.…”

“…We test our network on top of both RGB frames and optical flows. The optical flow is computed using the approach of [Brox et al, 2004]. As point action in the UTI dataset is a single action, we duplicate the image as input for the networks that require both subjects, i.e., the naive fusion network, the coupled network and the tri-coupled network.…”

Action recognition on video using recurrent neural networks

“…В работе [4] удалось получить точность классификации на уровне 52,75% с помощью k-NN классификатора и 73,25% при использовании системы опорных векторов (SVM-based). В работе [5] были получены схожие результаты, однако, в качестве классификатора использовались РНС, что позволило улучшить точность классификации до 74%.…”