In recent years, artificial intelligence (AI) has witnessed significant advancements in the field of education, with its ability to personalize and adapt content to individual student needs. In parallel, virtual reality (VR) has emerged as a powerful tutorial tool, providing immersive and interactive experiential learning experiences which has the advantages of improving students' motivation and engagement. Previous researchers have demonstrated the potential of ML algorithms, particularly RL, for generating educational content and VR environments. To create high-quality content, researchers have started exploring the integration of Machine Learning (ML) and Reinforcement Learning (RL) algorithms into Procedural Content Generation (PCG) methods for automatically generating both textual and non-textual content such as practice questions, quizzes, VR learning environments, etc., which have the potential to increase the efficiency and effectiveness of educational interventions. Nonetheless, the development of these techniques requires addressing several challenges. Significant advancements are yet to be made in developing and refining these algorithms to produce high-quality and effective educational content for VR applications. This article provides a comprehensive overview of the current state of research in reinforcement AI learning content generation for VR educational applications. For each area, it discusses the state-of-the-art techniques, applications, limitations, and challenges faced in development, covering the use of natural language processing, reinforcement learning, and machine learning algorithms. The review concludes by highlighting some of the key opportunities for future research in this field, including the development of more sophisticated models and the exploration of new applications of machine learning in educational technology.