The quality of boxing video is affected by many factors. For example, it needs to be compressed and encoded before transmission. In the process of transmission, it will encounter network conditions such as packet loss and jitter, which will affect the video quality. Combined with the proposed nine characteristic parameters affecting video quality, this paper proposes an architecture of video quality evaluation system. Aiming at the compression damage and transmission damage of leisure sports video, a video quality evaluation algorithm based on BP neural network (BPNN) is proposed. A specific Wushu video quality evaluation algorithm system is implemented. The system takes the result of feature engineering of 9 feature parameters of boxing video as the input and the subjective quality score of video as the training output. The mapping relationship is established by BPNN algorithm, and the objective evaluation quality of boxing video is finally obtained. The results show that using the neural network analysis model, the characteristic parameters of compression damage and transmission damage used in this paper can get better evaluation results. Compared with the comparison algorithm, the accuracy of the video quality evaluation method proposed in this paper has been greatly improved. The subjective characteristics of users are evaluated quantitatively and added to the objective video quality evaluation model in this paper, so as to make the video evaluation more accurate and closer to users.