Reinforcement learning is considered as a promising direction for driving policy learning. However, training autonomous driving vehicle with reinforcement learning in real environment involves non-affordable trial-and-error. It is more desirable to first train in a virtual environment and then transfer to the real environment. In this paper, we propose a novel realistic translation network to make model trained in virtual environment be workable in real world. The proposed network can convert non-realistic virtual image input into a realistic one with similar scene structure. Given realistic frames as input, driving policy trained by reinforcement learning can nicely adapt to real world driving. Experiments show that our proposed virtual to real (VR) reinforcement learning (RL) works pretty well. To our knowledge, this is the first successful case of driving policy trained by reinforcement learning that can adapt to real world driving data.Autonomous driving aims to make a vehicle sense its environment and navigate without human input. To achieve this goal, the most important task is to learn the driving policy that automatically outputs control signals for steering wheel, throttle, brake, etc., based on observed surroundings. The straight-forward idea is end-to-end supervised learning [3,4], which trains a neural network model mapping visual input directly to action output, and the training data is labeled image-action pairs. However, supervised approach usually requires large amount of data to train a model [31] that can generalize to different environments. Obtaining such amount of data is time consuming and requires significant human involvement. By contrast, reinforcement learning learns by a trial-and-error fashion, and does not require explicit supervision from human. Recently, reinforcement learning has been considered as a promising technique to learn driving policy due to its expertise in action planing [15,23,25].However, reinforcement learning requires agents to interact with environments, and undesirable driving actions would happen. Training autonomous driving cars in real world will cause damages to vehicles and the surroundings. Therefore, most of current research in autonomous driving with reinforcement learning focus on simulations [15,18,25] rather than training in real world. While an agent trained with reinforcement learning achieves

No abstract

Deep reinforcement learning has recently made significant progress in solving computer games and robotic control tasks. A known problem, though, is that policies overfit to the training environment and may not avoid rare, catastrophic events such as automotive accidents. A classical technique for improving the robustness of reinforcement learning algorithms is to train on a set of randomized environments, but this approach only guards against common situations. Recently, robust adversarial reinforcement learning (RARL) was developed, which allows efficient applications of random and systematic perturbations by a trained adversary. A limitation of RARL is that only the expected control objective is optimized; there is no explicit modeling or optimization of risk. Thus the agents do not consider the probability of catastrophic events (i.e., those inducing abnormally large negative reward), except through their effect on the expected objective. In this paper we introduce risk-averse robust adversarial reinforcement learning (RARARL), using a risk-averse protagonist and a risk-seeking adversary. We test our approach on a self-driving vehicle controller. We use an ensemble of policy networks to model risk as the variance of value functions. We show through experiments that a risk-averse agent is better equipped to handle a risk-seeking adversary, and experiences substantially fewer crashes compared to agents trained without an adversary.

We present a novel Metropolis-Hastings method for large datasets that uses small expected-size minibatches of data. Previous work on reducing the cost of Metropolis-Hastings tests yields only constant factor reductions versus using the full dataset for each sample. Here we present a method that can be tuned to provide arbitrarily small batch sizes, by adjusting either proposal step size or temperature. Our test uses the noise-tolerant Barker acceptance test with a novel additive correction variable. The resulting test has similar cost to a normal SGD update. Our experiments demonstrate several orderof-magnitude speedups over previous work.

No abstract