One of the significant courses in Chinese universities is English. This course is usually taught by a foreign language instructor. There will, however, necessarily be some communication hurdles between “foreign language teachers” and “native students.” This research presents an emotion recognition method for foreign language teachers in order to eliminate communication barriers between teachers and students and improve student learning efficiency. We discovered four factors of emotion recognition through literature analysis: smile, eye contact, gesture, and tone. We believe that differences in foreign language teachers’ performance in these four areas will have an impact on students’ emotion recognition and, as a result, on their learning efficiency. The influence of the foreign language teacher’s eye contact and gestures is larger (the weight of a single variable accounted for 30% or more) in the decision whether can improve the students’ classroom learning efficiency, according to 43 of the questionnaire data analysis. The second is the tone and smile (the weight of a single variable accounted for between 10 and 20%). Our research contributes to the body of knowledge on emotion recognition in university foreign language teachers by presenting a practical method for recognizing emotion in foreign language teachers. We recommend that college foreign language teachers pay attention to eye and gesture communication with students in English classrooms based on the findings. By enriching the style of emotional expression in class, college language teachers, particularly foreign language teachers, can improve communication, and connection with students.