With the informatization of education, the traditional English teaching evaluation method is increasingly unable to meet the needs of cultivating talents in colleges and universities. In this paper, we constructed an informatized English teaching evaluation system using two modules: students’ classroom learning status evaluation and teachers’ teaching quality evaluation. Using informationized face detection and expression recognition technology, the evaluation axis of students’ classroom learning status is established as “students’ head-up rate - students’ expression - students’ concentration - classroom learning status”. The fuzzy comprehensive judgment model based on a genetic algorithm was established by constructing a judgment matrix and calculating the weights of teaching quality evaluation indexes using a hierarchical analysis method. The experimental and control groups analyzed the English teaching evaluation system’s application. Regarding English achievement, the post-experimental scores of the students in the experimental class were 8.57 points higher than those of the control class, with a highly significant difference (P<0.01). In terms of student literacy, students in the experimental class showed significant improvement in four indicators, including learning motivation, learning situation and cognition, and their attitudes toward the quality of English teaching were in the range between satisfaction and comparative satisfaction, and the overall performance of the application practice of the English education evaluation system was good.