Stress is considered by many studies to affect traffic safety, and many researchers have attempted to monitor the dynamics of driving stress. Previous research has relied excessively on the positive effects of psychological indicators to improve the accuracy of stress monitoring models. However, psychological data collection sensors have not been widely used in conventional vehicles, which makes it impossible to apply the results of that research to actual driving tasks on a daily basis, even if the accuracy is high. This study designs a real driving task to extract data and proposes a driver's driving stress monitoring model based on driving behaviour, driving environment, and route familiarity. The driving behaviour is described by the speed and acceleration of the vehicle, and the driving environment is quantified by a dilated residual networks (DRN) model thazt divides the video image from the full region into subregions according to the distribution of the driver's attention. Based on the psychological data and driver stress inventory (DSI) results, the study used a K-means 3D cluster analysis to obtain the evaluation method of driving stress and constructed an extreme gradient boosting (XGBoost) model to monitor driving stress. Comparisons of performance with other models show that the XGBoost model significantly outperforms the other three mainstream machine learning algorithms and exceeds most traditional models without the use of psychological data. The model's performance indicators, accuracy, sensitivity, and precision, reached 91.18%-93.25%, 84.13%-89.37%, and 90.25%-91.34%, respectively. The study also summarises the ranking of effects of different scene elements on driving stress for each visual field. The results could make it possible to apply stress monitoring on a large scale to real driving situations, providing urban designers with advice on how to reduce driver stress and directing their attention to those visual areas and visual scene elements that have a higher impact on driving stress and need improvement.