As Taekwondo is widely accepted, there is a rapid increase in Taekwondo learners. Traditional Taekwondo teaching mode is challenging to meet the increasing educational needs. In Taekwondo teaching, in addition to teaching the basic skills of combat, Taekwondo Poomsae teaching is also required. The quality of Taekwondo Poomsae learning is critical to the overall strength of Taekwondo. For a long time, the judgment of the quality of Poomsae action has relied on the manual evaluation of teachers, which could be more conducive to forming an objective and accurate quality score. Therefore, it is necessary to introduce in-depth learning to find new solutions to these problems. First, a multi-sensor data fusion method is proposed to collect Taekwondo Poomsae action. Then, a Taekwondo Poomsae expertise integrated multiview feature extraction method is proposed. Finally, Convolutional Neural Network (CNN)-Mogrifier Long Short-Term Memory (LSTM) is proposed to train the generated Taekwondo Poomsae action scoring model, identifying whether Taekwondo Poomsae action meets the standard and improve Taekwondo teaching. The effect test of Taekwondo Poomsae action intelligent evaluation shows that the results of the proposed method follow the evaluation given by the teachers, and the discrimination of the scores is moderate, which indicates that the proposed method has good Taekwondo Poomsae action quality evaluation ability. Furthermore, the experimental results show that CNN-Mogrifier LSTM achieves high recognition accuracy of Taekwondo Poomsae, gives comprehensive evaluation and improvement suggestions for action completion quality, breaks away from the restrictions of teachers and venues, and realizes automatic and smart Taekwondo teaching.