NPU RGBD Dataset and a Feature-Enhanced LSTM-DGCN Method for Action Recognition of Basketball Players+

Abstract

Computer vision-based action recognition of basketball players in basketball training and competition has gradually become a research hotspot. However, owing to the complex technical action, diverse background, and limb occlusion, it remains a challenging task without effective solutions or public dataset benchmarks. In this study, we defined 32 kinds of atomic actions covering most of the complex actions for basketball players and built the dataset NPU RGB+D (a large scale dataset of basketball action recognition with RGB image data and Depth data captured in Northwestern Polytechnical University) for 12 kinds of actions of 10 professional basketball players with 2169 RGB+D videos and 75 thousand frames, including RGB frame sequences, depth maps, and skeleton coordinates. Through extracting the spatial features of the distances and angles between the joint points of basketball players, we created a new feature-enhanced skeleton-based method called LSTM-DGCN for basketball player action recognition based on the deep graph convolutional network (DGCN) and long short-term memory (LSTM) methods. Many advanced action recognition methods were evaluated on our dataset and compared with our proposed method. The experimental results show that the NPU RGB+D dataset is very competitive with the current action recognition algorithms and that our LSTM-DGCN outperforms the state-of-the-art action recognition methods in various evaluation criteria on our dataset. Our action classifications and this NPU RGB+D dataset are valuable for basketball player action recognition techniques. The feature-enhanced LSTM-DGCN has a more accurate action recognition effect, which improves the motion expression ability of the skeleton data.

An overview of Human Action Recognition in sports based on Computer Vision

“…In [ 139 ], the authors created a new feature-enhanced skeleton-based method called LSTM-DGCN for basketball player action recognition based on LSTM and the deep graph convolutional network (DGCN) methods. They extracted the spatial features of the distances and angles between the joint points of basketball players, and built a large-scale dataset of 12 complex actions (32 kinds of atomic actions) for basketball players with RGB image data and Depth data captured at Northwestern Polytechnical University, named NPU RGB + D. The dataset consists of 2169 videos, i.e., 75 thousand frames, including RGB frame sequences, depth maps, and skeleton coordinates.…”

A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications

“…NPUBasketball [67] is composed of 2,169 self-recorded video clips of basketball actions performed by professional players and each video belongs to one of 12 categories: standing dribble, front dribble, moving dribble, cross-leg dribble, behind-the-back dribble, turning around, squat, run with ball, overhead pass (catch or shoot), one-hand shoot, chest pass (catch or shoot), and side throw. Different from FineBasketball and SpaceJam, NPUBasketball provides not only RGB frames, but also depth maps and skeleton of players, thus, it can be used for developing various types of action recognition models.…”

Real-Time Recognition Method of Video Basketball Technical Action Based on Target Detection Algorithm

“…In the formula, q � q u 􏼈 􏼉 u�1,2,...,m represents the histogram model, p u(y) 􏽮 􏽯 u�1,2,...,m describes the candidate target model, and y describes the coordinates corresponding to the center of the candidate target area [15][16][17]. The target area is usually rectangular and oval.…”