Autonomous Curiosity for Real-Time Training Onboard Robotic Agents

Abstract

Learning requires both study and curiosity. A good learner is not only good at extracting information from the data given to it, but also skilled at finding the right new information to learn from. This is especially true when a human operator is required to provide the ground truth-such a source should only be queried sparingly. In this work, we address the problem of curiosity as it relates to online, real-time, human-in-the-loop training of an object detection algorithm onboard a robotic platform, one where motion produces new views of the subject. We propose a deep reinforcement learning approach that decides when to ask the human user for ground truth, and when to move. Through a series of experiments, we demonstrate that our agent learns a movement and request policy that is at least 3x more effective at using human user interactions to train an object detector than untrained approaches, and is generalizable to a variety of subjects and environments.

Learning to Learn in Simulation

“…In prior work (Teng and Iannucci 2019), we primarily examined the improvement garnered from a single user interaction. We define a training episode as a single training session with one target object.…”

“…where P i is the performance (i.e., the AP) of the trainee model at step i, and P 0 is reserved for the performance of the object detection model before any online training (Teng and Iannucci 2019). In this paper, we will use the final…”

“…The curiosity agent receives feedback from the trainee, which consists of the object detector being trained and an object tracker which produces training events as long as the target is tracked. It is responsible for moving the drone to get a better view, if necessary, or asking the human user for ground truth (Teng and Iannucci 2019) they can from the information they have, and seek new information as needed. They may have guidance from a teacher or mentor, but this guidance is limited and should be used sparingly.…”

“…In our prior work (Teng and Iannucci 2019), we introduced autonomous curiosity, a way for a pre-trained curiosity agent to move a drone around to increase how beneficial (the training benefit) each ground truth user interaction is in training the drone's model. Figure 1 shows how our system interacts with both the drone's navigational system as well as the human user.…”

“…In this paper, we expand the problem setup of autonomous curiosity to two dimensions and four movement actionstowards and away from the subject in addition to orbital movement. This introduces an additional consideration-we cannot make the assumption that all motions of the drone take the same amount of time (Teng and Iannucci 2019). We hypothesize that agents trained without a notion of the timecost of each of its actions, on the specific robotic platform they will be deployed, will not perform as well as one which was trained with this in mind.…”