We aimed to clarify the relationship between Chinese adolescents’ physical activity and academic achievement, the mediating role of physical and mental health, and provide a scientific reference for the integration of sports and education. Based on baseline data from the China Education Panel Survey, we conducted a nationwide study of 12,960 adolescents. We used nonlinear models, mediation effect analysis, and other measurement methods. Three significant findings emerged: (1) adolescent physical activity duration and academic achievement showed an inverted U-shaped relationship. Moderate physical activity improves academic achievement. The inverted U-shaped inflection point is about 3.549 h/week; (2) physical activity among the male group has a stronger effect on promoting academic achievement, while the positive effect of physical activity among females is not significant. The difference in effectiveness between urban and rural youth groups is relatively small, but the inflection point of rural youth groups is higher, and (3) the level of physical and mental health effectively mediates the inverted U-shaped influence of youth physical activity on academic achievement. We recommend that governments and schools should implement physical activity appropriately and suggest the feasibility of using physical activity to narrow the growth gap between urban and rural youth.