This study takes environmental factors and individual factors as variables to explore the deep internal mechanism of the impact of a comprehensive environment on higher education physical education (PE) teachers’ job burnout. Little research has been done on how environmental factors affect the internal mechanism of college and university PE teachers’ job burnout through individual factors (e.g., professional pressure and teaching efficacy). In this study, the participants were 231 PE teachers from seven comprehensive universities, and four questionnaires were administered to measure the participants’ job burnout, perceived overall environment, teaching efficacy, and occupational stress. Research has found that environmental factors have a significant negative impact on occupational stress, and occupational stress plays an important mediating role between environment and occupational burnout. Research has shown that differences in external environments lead to varying levels of personal stress among college physical education teachers, which in turn affects their level of occupational burnout. The study concludes that a good social, working, and living environment helps to reduce the work pressure on PE teachers, improves their sense of teaching efficacy, and inhibits the occurrence of teachers’ job burnout.