VR Computer Role-Playing Game Using Reinforcement Learning

Abstract

This paper shall discuss the process of making one of the first Computer Role-Playing Virtual Reality Games that take advantage of the underutilized hand-tracking technology provided by the Oculus Quest 2 headset. The game takes place in two separate fictional dystopian worlds. The player will have to navigate and explore to discover its secrets while trying to survive against enemies that employ a reinforcement learning algorithm to stop the player. Through extensive experimentation and fine-tuning of various implementation aspects, including positive and negative rewards, hyper parameter values related to Proximal Policy Optimization (PPO), and the number of environment observations, our agent achieved highly favorable outcomes in terms of general machine learning model efficiency indicators. As explained in the methodology section, the model's behavior aligns with the desired behavior, achieved by implementing three rays at different angles to determine some of the rewards. These results validate the effectiveness and success of our approach in training the agent to exhibit desired behaviors and achieve desirable outcomes.