Currently, there are two main problems with the evaluation of college English teaching quality: the lack of a complete evaluation system, and the limited quantification of evaluation indices. To solve the problems, this paper sums up the evaluation contents of college English teaching quality, and identifies the factors affecting the teaching quality of college English. On this basis, an improved evaluation system was established for the assessment of college English teaching quality, which is theoretically innovative. Next, grey clustering analysis and entropy weight method were combined into a robust evaluation model for college English teaching quality. This research provides a new and applicable solution to quantification of college English teaching quality.