HKGAIL: Policy shaping via integrating human knowledge with generative adversarial imitation learning

Abstract

Generative adversarial imitation learning (GAIL) has shown good results in several research areas by taking advantage of generative adversarial networks. However, GAIL lacks a reward mechanism and usually adopts a model-free approach based on stochastic policies, which is not ideal for solving complex, dynamically uncertain population intelligence problems, especially in the face of autonomous driving environments. In this paper, a policy framework is shaped by combining the human knowledge with GAIL (HKGAIL). HKGAIL embeds human decision models into the learning process to infer the underlying structure of expert demonstrations. The skills learned from expert demonstrations can directly guide the actions (policies) of the learning process of the agents, and the policies can be optimized through the feedback function of the discriminator. Experiments on both driving and landing tasks show that HKGAIL was able to better fit the policy close to the expert, and was 16.2% safer than GAIL for the driving task.