Adaptive learning is an important part of Intelligent Tutoring System (ITS). Given that students have different learning targets and knowledge concepts proficiency, a smart intelligent tutor should be able to provide personalized learning materials to them, and help students master target knowledge and skills with learning materials as less as possible. Reinforcement Learning (RL) algorithms are good at solving sequence decision problems, so they are widely used in learning material recommendation. However, the existing intelligent tutoring systems based on reinforcement learning usually consider only one learning target. Moreover, the agent needs to learn in the case of sparse rewards, resulting in inefficient learning. To this end, we propose a curriculumoriented multi-goal reinforcement learning method, which combines an off-policy RL algorithm with Hindsight Experience Replay (HER) to enable the agent to learn from past failed experiences to alleviate the problem of sparse rewards. Besides, our method is applicable to the case of multi-goal learning, and the agent learns specific strategy for each goal. Additionally, according to different learning stages of the agent, we set different learning pseudo goals adaptively for it to accelerate learning speed.