Traditional English teaching model neglects student emotions, making many tired of learning. Machine learning supports end-to-end recognition of learning emotions, such that the recognition system can adaptively adjust the learning difficulty in English classroom. With the help of machine learning, this paper presents a method to extract the facial expression features of students in business English class, and establishes a student emotion recognition model, which consists of such modules as emotion mechanism, signal acquisition, analysis and recognition, emotion understanding, emotion expression, and wearable equipment. The results show that the proposed emotion recognition model monitors the real-time emotional states of each student during English learning; upon detecting frustration or boredom, machine learning will timely switch to the contents that interest the student or easier to learn, keeping the student active in learning. The research provides an end-to-end student emotion recognition system to assist with classroom teaching, and enhance the positive emotions of students in English learning.