Rapid growth and development of civil engineering in recent years inspire building enterprises to concentrate on construction contractor selection for achieving more construction quality and lower construction cost. The existing studies generally regard the process of selecting the best contractor as a multi-criteria group decision making problem. Few research studies addressed the contractor selection problem in the context of large-scale group decision making, which is common in practical scenarios in terms of major construction projects as a number of experts with diverse backgrounds are usually involved. On this basis, we establish a contractor selection framework under large-scale group decision making environment, which covers expert classification, consensus reaching process, collective decision matrix generation, and the ranking-oriented decision making method. We cluster expert group with K-means clustering method based on expertise levels, which are depicted by six features generated with an expertise identification approach. The consensus model manages consensus reaching process from both intra-and interlayers and takes into account the interactions between them. After reaching agreements among experts, this paper utilizes the concept of proportional hesitant fuzzy linguistic term set to assemble intra-subgroup assessments for the reduction of information loss or distortion. Then, an aggregation process carries on as to gather subgroup assessments in which the subgroup weights are derived from their cluster centers and sizes in the use of the TOPSIS method. Finally, the well-established decision making tool integrating qualitative and quantitative criteria, ELECTRE III, is adapted to elicit the ranking of bidders. An illustrative study and a comparative analysis are performed to demonstrate the feasibility and effectiveness of the established multi-criteria group decision making approach.