Redirection Controller Using Reinforcement Learning

Abstract

There is a growing demand for redirected walking (RDW) techniques and their application to various virtual and real environments. To apply appropriate RDW methods and manipulation according to the real and virtual environments, the RDW controllers are predominantly used. There are three types of RDW controllers: direct scripted controller, generalized controller, and predictive controller. The scripted controller type pre-scripts the mapping between the real and virtual environments. The generalized controller type employs the RDW method and manipulation quantities according to a certain procedure depending on the user's position in relation to the real space. This approach has the potential to be reused in any environment because it does not require pre-scripting of user movements; however, it is not fully optimized. The predictive controller type predicts the user's future path using the user's behavior and manages RDW techniques. This approach is highly anticipated to be very effective and versatile; however, it has not been sufficiently developed. This paper proposes a novel RDW controller using reinforcement learning (RL) with advanced plannability/versatility. Our simulation experiments indicate that the proposed method can reduce the number of reset manipulations, which is one of the indicators of the effectiveness of the RDW controller, compared to the generalized controller under real environments with many obstacles. Meanwhile, the experimental results also showed that the gain output by the proposed method oscillates. The results of a user study conducted showed that the proposed RDW controller can reduce the number of resets compared to the conventional generalized controller. Furthermore, no adverse effects such as cybersickness associated with the oscillation of the output gain were evinced. The simulation and user studies demonstrate that the proposed RDW controller with RL outperforms the existing generalized controllers and can be applied to users as it only causes cybersickness comparable to that in the case of conventional generalized controllers.INDEX TERMS Heuristic controller, redirected walking, reinforcement learning, virtual reality.

MARR : A Multi-Agent Reinforcement Resetter for Redirected Walking

“…Various reinforcement learning algorithms have been proposed in the reinforcement learning community, such as Proximal Policy Optimization (PPO) [36] and Soft Actor-Critic (SAC) [37]. Redirection controllers trained with the proposed reinforcement learning algorithm are also being actively studied [24], [32], [34], [25], [38], [39], [40], [41].…”

Adaptive Optimization Algorithm for Resetting Techniques in Obstacle-Ridden Environments

“…By employing Proximal Policy Optimization (PPO), a deep neural network is trained to directly estimate the rotation, translation, and curvature gains to transform a virtual environment, given a user's position and orientation in the tracked space. Another work by Chang et al [9] presented a new redirection controller by PPO, and applied a new RDW algorithm (advanced center-based translation gain) and a new reset method (Turn-to-Furthest) for experimental comparison.…”

Optimal Planning for Redirected Walking Based on Reinforcement Learning in Multi-user Environment with Irregularly Shaped Physical Space

“…Various reinforcement learning algorithms have been proposed in the reinforcement learning community, such as Proximal Policy Optimization (PPO) [30] and Soft Actor-Critic (SAC) [9]. Redirection controllers trained with the proposed reinforcement learning algorithm are also being actively studied [5,10,17,18,33].…”