School recess is a daily opportunity for school-age students to be physically active. However, in some territories teachers often use recess for other purposes (e.g., children’s poor classroom behavior might be punished with reduced time for recess). This study aimed to examine the impact of such practices on children’s physical activity (PA) and the relationships between PA, gender, body mass index (BMI), and academic achievement. Forty-six first-grade students from two natural classrooms wore an accelerometer over the course of 6 weeks to measure their metabolic equivalent of task (METs) and sedentary behavior during school recess. Gender, age, BMI, the classroom to which students belonged, and academic achievement were also analyzed in two Generalized Estimating Equations models. Results revealed that boys achieved more METs and spent less time participating in sedentary behavior than girls during recess. Children within a healthy weight range of BMI yielded more METs than underweight and overweight/obese children. Academic achievement was positively associated with the METS and negatively with the sedentary behavior. Finally, withholding all or part of school recess significantly reduced children’s PA and extended their sedentary behavior. The literature indicates that school recess plays an important role in promoting numerous children’s health outcomes. Therefore, students should not be excluded from participation in all or part of recess.