Evaluating the public welfare level of the urban rail transit systems has not only great significance for the government to provide fair and reasonable subsidies but for the better operation and management of urban rail transit enterprises. An evaluation index system composed of 3 criterion-level indicators and 12 subindexes has been established in this paper. The 3 criterion-level indicators conclude service level, social benefit, and policy loss which are all affected by operation management. Besides, the subjective and objective comprehensive weighting method combined with the analytic hierarchy process method and the entropy weight method is proposed to calculate the index-level weights. Furtherly, the grey correlation-TOPSIS comprehensive evaluation method was designed to calculate the comprehensive evaluation value of the public welfare level of each city. To verify the effectiveness of the proposed method, urban rail transit systems in 16 Chinese cities are studied as a case study. The results show that (1) the three indicators of passenger travel cost (25.69%), the increase in housing prices around urban rail transit(10.74%), and operating cost ratio(9.95%) are more likely to affect the evaluation of public welfare level of urban rail transit. (2) the level of public welfare in different cities is not balanced. The cities with a relatively high level of public welfare relative closeness exceeding 0.5 include Shanghai, Beijing, Shenzhen, Guangzhou, Suzhou, Wuhan, Nanjing, Wuxi, and Dalian. (3) Both GDP and urban population are positively correlated with the relative closeness of social benefit and service level. (4) The level of public welfare can be improved by reducing the fare price and improving the service level, such as increasing the network density, reducing the departure interval, and increasing the average speed.