With the rapid development of deep learning, computer vision has also become a rapidly developing field in the field of artificial intelligence. Combining the physical training of deep learning will bring good practical value. Physical training has different effects on people’s body shape, physical function, and physical quality. It is mainly reflected in the changes of relevant physical indicators after physical training. Therefore, the purpose of this article is to study the method of evaluating the impact of sports training on physical indicators based on deep learning. This paper mainly uses the convolutional neural network in deep learning to design sports training, then constructs the evaluation system of physical index impact, and finally uses the deep learning algorithm to evaluate the impact of physical index. The experimental results show that the accuracy of the algorithm proposed in this paper is significantly higher than that of the other three algorithms. Firstly, in the angular motion, the accuracy of the mean algorithm is 0.4, the accuracy of the variance algorithm is 0.2, the accuracy of the RFE algorithm is 0.4, and the accuracy of the DLA algorithm is 0.6. Similarly, in foot racing and skill sports, the accuracy of the algorithm proposed in this paper is significantly higher than that of other algorithms. Therefore, the method proposed in this paper is more effective in the evaluation of the impact of physical training on physical indicators.