With the rapid development of computer vision technology, human action recognition technology has occupied an important position in this field. The basic human action recognition system is mainly composed of three parts: moving target detection, feature extraction, and human action recognition. In order to understand the action signs of gymnastics, this article uses network communication and contour feature extraction to extract different morphological features during gymnastics. Then, the finite difference algorithm of edge curvature is used to classify different gymnastic actions and analyze and discuss the Gaussian background. A modular method, an improved hybrid Gaussian modeling method, is proposed, which adaptively selects the number of Gaussian distributions. The research results show that, compared with traditional contour extraction, the resolution of gymnastic motion features extracted through network communication and body contour features is clearer, and the increase rate is more than 30%. Moreover, the method proposed in this paper removes noise in the image extraction process, the effect is good, and the athlete’s action marks are very clear, which can achieve the research goal.