Spotting Football Events Using Two-Stream Convolutional Neural Network and Dilated Recurrent Neural Network

Abstract

This paper addresses the problem of event detection and localization in long football (soccer) videos. Our key idea is that understanding the long-range dependencies between video frames is imperative for accurate event localization in long football videos. Additionally, proper event detection is not likely for fast movements in football videos without considering mid-range and short-range correlations between neighboring video frames. We argue that event spotting can be considerably improved by considering short-range to long-range frame dependencies in a unified architecture. To model long-range and midrange dependencies, we propose to use the dilated recurrent neural network (DilatedRNN) with long shortterm memory (LSTM) units, grounded on two-stream convolutional neural network (Two-stream CNN) features. While two-stream CNN extracts local spatiotemporal features necessary for fine-level details, the DilatedRNN makes the information obtained from distant frames available for the classifier and spotting algorithms. Evaluating our event spotting algorithm on the largest publicly available benchmark football dataset -SoccerNet-shows an accuracy improvement of 0.8% -13.6% compared to state of the art, and up to 30.1% accuracy gain in comparison to the baselines. We also investigate the contribution of each neural network component in spotting accuracy through an extensive ablation study.INDEX TERMS Deep learning, Dilated RNNs, Sport videos event detection, Two-stream CNNs.

[Retracted] Design and Simulation of Human Resource Allocation Model Based on Double‐Cycle Neural Network

“…The process design of the recommendation algorithm should also be tailored to different application scenarios and data characteristics. Currently used algorithms in the HR field are usually statistical algorithms that do not take into account the hidden features of the data and rely on simple scoring and expert judgment mechanisms, which can easily lead to information scarcity problems [ 28 ].…”

An overview of Human Action Recognition in sports based on Computer Vision

“…Also, for the same problem, authors in [ 121 ] proposed the dilated recurrent neural network (DilatedRNN) with LSTM units, grounded on Two-stream CNN features to model long-range and mid-range dependencies. The Two-stream CNN extracts local Spatio-temporal features, and the DilatedRNN makes the information obtained from distant frames available for the classifier and spotting algorithms.…”

A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications

“…2011-2016 2017-present Football [37]- [42] [30], [43]- [52] Basketball [53]- [59] [60]- [72] Volleyball [73]- [77] [78]-[83] Hockey [84]- [89] [90]-[99] Diving [100] [101]-[107] Tennis [108]- [113] [114]- [123] Table tennis [124]- [129] [130]-[138] Gymnastics [139]- [144] [145]-[148] Badminton [149]- [154] [155]- [164] Figure Skating [165], [166] [2], [167]- [174] Recently, researchers in the communities of computer vision and sports pay much attention to sports video analysis, including building datasets and proposing novel methodologies [2], [17]- [30]. In most existing works on sports video analysis, recognizing the actions of players in videos is crucial.…”