The study aimed (a) to examine changes in physical activity (PA) during the whole day, school hours, recess, and physical education classes (PEC) during a 2‐year period in primary and secondary students; (b) to identify changes in the proportion of compliance with specific PA recommendations for these periods; and (c) to examine whether PA levels at baseline are associated with PA levels 2 years later. Eight hundred and fourteen (51.8% boys) children and 658 (50.1% boys) adolescents from 41 Spanish schools participated in the study. Hip‐worn accelerometers were used to assess PA during different time periods. Light PA (LPA) declined during the whole day, school hours, recess (all P < 0.001, except child girls for recess), and PEC (all, P < 0.05) in children and adolescents. Moderate‐to‐vigorous PA (MVPA) during the whole day and recess declined in child boys (P < 0.01 and P < 0.001, respectively) and adolescent boys (P < 0.001 and P < 0.05, respectively). MVPA during PEC declined in adolescent boys (P < 0.001) and adolescent girls (all P < 0.05). The proportion of compliance with the specific PA recommendations for these periods declined (P < 0.05), except for PEC in adolescent girls. PA during the whole day at baseline was moderately associated with PA during the whole day years later (ICCs = 0.210‐0.544, with one exception), but this association was lower for the school‐based PA. In conclusion, time spent in MVPA and LPA during the whole day and recess declined over time in child and adolescent boys and during PEC in adolescents. These findings highlight the need to promote PA interventions in these settings.