Accelerating imitation learning through crowdsourcing

Abstract

Abstract-Although imitation learning is a powerful technique for robot learning and knowledge acquisition from naïve human users, it often suffers from the need for expensive human demonstrations. In some cases the robot has an insufficient number of useful demonstrations, while in others its learning ability is limited by the number of users it directly interacts with. We propose an approach that overcomes these shortcomings by using crowdsourcing to collect a wider variety of examples from a large pool of human demonstrators online. We present a new goal-based imitation learning framework which utilizes crowdsourcing as a major source of human demonstration data. We demonstrate the effectiveness of our approach experimentally on a scenario where the robot learns to build 2D object models on a table from basic building blocks using knowledge gained from locals and online crowd workers. In addition, we show how the robot can use this knowledge to support human-robot collaboration tasks such as goal inference through object-part classification and missing-part prediction. We report results from a user study involving fourteen local demonstrators and hundreds of crowd workers on 16 different model building tasks.

Machine learning from crowds: A systematic review of its applications

“…There are a great number of applications in other domains, such as security or energy, which are also relevant for this study (Table ): deduplication of digital libraries (Georgescu et al, ); imitation learning (Chung et al, ); evaluation of procedural content generation (Roberts & Chen, ); action model acquisition (Zhuo, ); weighting antivirus labels (Kantchelian et al, ); aerosol optical depth estimation (Djuric et al, ); point of interest labeling (Hu et al, ); detection of spatial events (Ouyang et al, ); interstate conflict measurement (D'Orazio et al, ); annotation of energy data (Cao et al, ); extracting semantic attributes to describe concepts (Tian et al, ); category learning (Danileiko & Lee, ); crowd databases (Robinson et al, ); and network quality measurements (Li, Gao, et al, ).…”

“…One of the most common needs revealed in the publications was to find some way to model the instance difficulty for a task (Aung & Whitehill, ; Cao et al, ; Chung et al, ; Duan et al, ; Nguyen‐Dinh et al, ; Ni et al, ; Wan & Aggarwal, ). Some authors (Chung et al, ) emphasize that the difficulty for a task could be used to reduce the number of annotations required, and hence the cost, for the correct coverage of an example, thus using fewer annotators for easy examples, while collecting more labels for the most difficult ones.…”

Unsupervised learning of multi-hypothesized pick-and-place task templates via crowdsourcing

“…al. [11]. While not explicitly dealing with WAPP tasks, the authors enable a robot to reconstruct 2D block patterns from a corpus of crowdsourced end-state configurations.…”

PlanIt: A crowdsourcing approach for learning to plan paths from large scale preference feedback

“…Similarly, we also learn from the preference data of non-expert users. However, we use crowdsourcing like Chung et al [33] for eliciting user feedback which allows us to learn from large amount of preference data. In experiments, we compare against Jain's trajectory preference perceptron algorithm.…”