With the development and maturity of the Internet education industry, more and more vocational colleges have opened English teaching courses based on massive open online courses courses. However, there are many English-related courses on the massive open online courses course platform, and the use of scientific recommendation models can improve the teaching quality of such courses. Therefore, this research attempts to design two improved attention mechanisms and user-based embedded expression using meta-path technology. At the same time, these two are combined with reinforcement learning technology to design an improved massive open online courses English course recommendation model. The test results show that the hit rate of the model designed in this study is 89.84%, 74.28%, 70.81% and 71.35% respectively when the rank number is 20 and the parameter is 10. At this time, the cumulative income of normalized discount is 48.24%, 34.58%, 25.96% and 28.69% respectively. However, when the number of calculated samples reaches the maximum value of 1158609, the calculation time of the improved reinforcement learning recommendation model is 1867 seconds, which is also higher than the comparison model. The experimental results show that the curriculum recommendation accuracy of the massive open online courses recommendation model designed in this study is higher and the recommendation results are more reasonable. The results of this research have a certain application potential in the field of the construction of online education in colleges and universities.