Green buildings are an important initiative to address energy and environmental issues in the construction field. The high-quality development of green buildings is affected by many factors, and it is necessary to identify the critical factors affecting the high-quality development of green buildings and analyze them systematically. The adopted literature analysis method and expert consultation method, the DIM (DEMATEL-ISM-MICMAC) model was established to explore critical factors influencing green buildings’ high-quality development and their internal hierarchical structure, interrelationships, and mechanisms. Then, targeted suggestions were put forward to promote green buildings’ high-quality development. The results showed that: (1) The critical factors influencing green buildings’ high-quality development could be divided into five levels, three groups, and four areas. The economic development level, living standard of residents, education level, incentive policies, and compulsory laws and regulations were in the deep factor group, fundamentally affecting green buildings’ high-quality development. (2) In terms of drive and centrality, the economic development level, living standard of residents, education level, and incentive policies were at the forefront, playing a vital role in the high-quality development of green buildings.