Prolonged fatigue among elementary and junior high school teachers not only damages their health but also affects the quality of education. The aim of this study was to determine the factors of working conditions associated with prolonged fatigue among teachers at public elementary and junior high schools. We distributed a self-reported, anonymous questionnaire to 3,154 teachers (1,983 in elementary schools, 1,171 in junior high schools) working in public schools in a city in Japan. They were asked to assess 18 aspects of their working conditions using a seven-point Likert scale. Prolonged fatigue was measured using the Japanese version of the checklist individual strength questionnaire. Multiple regression analysis was used to examine the association between working conditions and prolonged fatigue. Gender, age, and school type were introduced as confounders. In all, 2,167 teachers participated in this study. Results showed that qualitative and quantitative workload (time pressure due to heavy workload, interruptions, physically demanding job, extra work at home), communication with colleagues (poor communication, lack of support), and career factors (underestimation of performance by the board of education or supervisors, occupational position not reflecting training, lack of prospects for work, job insecurity) were associated with prolonged fatigue.