To address the issue of traditional static evaluation models being unable to comprehensively analyse the performance of ultra-supercritical coal-fired units under varying loads, we propose a dynamic comprehensive evaluation model based on the improved Criteria Importance Through Inter-criteria Correlation (CRITIC) method and Entropy Weight Method (EWM). The comprehensive performance evaluation index system of ultra-supercritical coal fired units is constructed by examining the boiler performance, turbine performance, plant power performance, environmental performance, and flexible performance of coal powered units. The CRITIC and EWM methods are used to calculate the weights of the indicators, which are then combined with the static evaluation results. Using a dynamic comprehensive evaluation model, we analysed ultra-supercritical coal-fired units, taking into account time weight. This allowed us to obtain the comprehensive dynamic real-time evaluation value of the units under different loads. The research indicates that the dynamic comprehensive evaluation model, which uses an improved CRITIC and EWM, has significant advantages in improving evaluation accuracy, weight-balanced distribution, and generality. Provides more accurate, reasonable, and reliable evaluation results for practical decision making.