In this paper, we describe results of application of reinforcement learning on full body control of a humanoid robot. We start with a simple task of achieving vertical position of robot's torso. We use an actor-critic neural network architecture, which is well established approach for reinforcement learning continuous action policies. Our experimental setup includes an instance of the NAO robot in the Webots simulation environment and custom adaptation components for Keras-rl reinforcement learning framework. We present minor modifications of original algorithm and discuss several encountered challenges of applying deep reinforcement learning methods to humanoid robot control.

Reinforcement learning is one of the artificial intelligence methods that enable robots to judge and operate situations on their own by learning to perform tasks. Previous reinforcement learning research has mainly focused on tasks performed by individual robots; however, everyday tasks, such as balancing tables, often require cooperation between two individuals to avoid injury when moving. In this research, we propose a deep reinforcement learning-based technique for robots to perform a table-balancing task in cooperation with a human. The cooperative robot proposed in this paper recognizes human behavior to balance the table. This recognition is achieved by utilizing the robot’s camera to take an image of the state of the table, then the table-balance action is performed afterward. Deep Q-network (DQN) is a deep reinforcement learning technology applied to cooperative robots. As a result of learning table balancing, on average, the cooperative robot showed a 90% optimal policy convergence rate in 20 runs of training with optimal hyperparameters applied to DQN-based techniques. In the H/W experiment, the trained DQN-based robot achieved an operation precision of 90%, thus verifying its excellent performance.