Driver fatigue level was considered an accumulated result contributed by circadian rhythms, hours of sleep before driving, driving duration, and break time during driving. This article presents an investigation into the regression model between driver fatigue level and the above four time-related variables. With the cooperation of one commercial transportation company, a Naturalistic Driving Study (NDS) was conducted, and NDS data from thirty-four middle-aged drivers were selected for analysis. With regard to the circadian rhythms, commercial drivers operated the vehicle and started driving at around 09:00, 14:00, and 21:00, respectively. Participants’ time of sleep before driving is also surveyed, and a range from 4 to 7 hours was selected. The commercial driving route was the same for all participants. After getting the fatigue level of all participants using the Karolinska Sleepiness Scale (KSS), the discrete KSS data were converted into consecutive value, and curve fitting methods were adopted for modeling. In addition, a linear regression model was proposed to represent the relationship between accumulated fatigue level and the four time-related variables. Finally, the prediction model was verified by the driving performance measurement: standard deviation of lateral position. The results demonstrated that fatigue prediction results are significantly relevant to driving performance. In conclusion, the fatigue prediction model proposed in this study could be implemented to predict the risk driving period and the maximum consecutive driving time once the driving schedule is determined, and the fatigue driving behavior could be avoided or alleviated by optimizing the driving and break schedule.