Towards robust and domain agnostic reinforcement learning competitions

Abstract

Reinforcement learning competitions have formed the basis for standard research benchmarks, galvanized advances in the state-of-the-art, and shaped the direction of the field. Despite this, a majority of challenges suffer from the same fundamental problems: participant solutions to the posed challenge are usually domain-specific, biased to maximally exploit compute resources, and not guaranteed to be reproducible. In this paper, we present a new framework of competition design that promotes the development of algorithms that overcome these barriers. We propose four central mechanisms for achieving this end: submission retraining, domain randomization, desemantization through domain obfuscation, and the © W.H. Guss et al.

Critic Guided Segmentation of Rewarding Objects in First-Person Views

“…We evaluate our approach in the first-person Minecraft environments from the MineRL 2020 Competition [4]. The provided imitation learning database consists of data recorded from human players, and we use only images and the recorded sparse reward signals from the database to train our model.…”

“…Such technique can contribute to explainable AI [3], symbolic and causal reasoning in the space of detected objects, robotics, or as auxiliary information [7] for an RL setup. Learning the optimal policy in sparse reward environments [4] is an important challenge in Deep Reinforcement Learning (DRL) [1,15,5]. A masking network for rewarding objects can support an actor-critic RL setup in a way that is intuitive and explainable to a human.…”

“…This work was motivated by the "MineRL NeurIPS 2020 Competition: Sample Efficient Reinforcement Learning in Minecraft" [4], where our AI agent won the 1st place. See Section 3 for details on the challenge, goals, and objects that an agent needs to learn in the world of Minecraft.…”