With the explosive growth of sports video data on the internet platform, how to scientifically manage this information has become a major challenge in the current big data era. In this context, a new lightweight player segmentation algorithm is proposed to realize the automatic analysis of basketball game video. Firstly, semantic events are expressed by extracting group and global motion features. A complete basketball game video is divided into three stages, and a basketball event classification method integrating global group motion patterns and domain knowledge is proposed. Secondly, a player segmentation algorithm based on lightweight deep learning is proposed to detect basketball players, segment the players, and finally extract players’ spatial features based on deep learning to realize players’ pose estimation. As the experimental results indicate, when a proposed 2-stage classification algorithm is used to classify the videos, the accuracy of identifying layup, the shooting, and other 2-pointers are improved by 21.26% and 6.41%, respectively. And the accuracy of average events sees an improvement of 2.74%. The results imply that the 2-stage classification based on event-occ is effective. After comparing the four methods of classifying players, it is found that there is no significant difference among these four methods about the accuracy of segmenting. Nevertheless, when judged with the time that these methods take separately, FCN-CNN (Fully Convolutional Network-Convolutional Neural Network) based on superpixels has overwhelming advantages. The event analysis method of basketball game video proposed here can realize the automatic analysis of basketball video, which is beneficial to promoting the rapid development of basketball and even sports.