Background. English is one of the courses offered in all colleges and universities. The quality of English teaching is directly related to the quality of talent training and the development of students themselves. “Teaching quality evaluation” specifically refers to the education evaluation with teaching as the evaluation object. It is the core and foundation of the whole education evaluation. Teaching quality evaluation is based on certain teaching objectives and teaching norms and standards, through the systematic detection and assessment of teaching and learning. Evaluate its teaching effect and the degree of realization of teaching objectives, and use scientific and feasible methods to make corresponding value judgments to improve the process of teaching. To improve the accuracy of English teaching ability evaluation, an English teaching ability evaluation algorithm based on frequency effect is proposed. Methods. The paper proposes an English teaching ability evaluation algorithm based on frequency effect. Firstly, it constructs the evaluation index system of English teaching ability, including expert evaluation system, student evaluation system, and teacher evaluation system. Then, the indexes affecting the evaluation of English teaching ability are quantified by fuzzy synthesis, and the evaluation indexes are refined. Finally, the basic principle of frequency effect is analyzed, combined with the convolutional neural network. Results. The convolutional neural network evaluation model is constructed, the teaching ability indicators are input into the model, the final evaluation results are output, and the design of the English teaching ability evaluation algorithm based on frequency effect is completed. Conclusions. The experimental results show that this method has high accuracy and efficiency.