Learning to Race Through Coordinate Descent Bayesian Optimisation

Abstract

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout its execution. In these cases, strategies to optimise control policies for individual stages of the process might not be applicable, and instead the whole policy might have to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. We propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.

Autonomous Racing with Multiple Vehicles using a Parallelized Optimization with Safety Guarantee using Control Barrier Functions

“…The learning-based controllers [15]- [17], [30] leverage the control input bounds to achieve optimal performance in iterative tasks. Model-free methods like Bayesian optimization (BO) [31], Gaussian processes (GPs) [9], deep neural networks (DNN) [32], [33] and deep reinforcement learning (DRL) [10], [34] have also been exploited to develop controllers that result in agile maneuvers for the ego car. To deal with other surrounding vehicles, DRL has also been used in [35] to control the ego vehicle during overtake maneuvers.…”