Study on Motion Forms of Mobile Robots Generated by Q-Learning Process Based on Reward Databases

Abstract

This paper investigates the motion forms of robots generated by the Q-Learning algorithm during the learning process. We analyzed the manner in which a caterpillar robot, which performs looping motions using two actuators, acquires advance actions by focusing on the process. By observing a series of processes, we confirmed that various motion forms appeared or disappeared as a result of their interactions with the learning process and approach an optimum motion form. In most algorithms, such motion forms cannot appear in the learning process because its framework is almost predetermined by the teacher data, and the cost functions for learning cannot be usually considered as a continuous process. The characteristics of reinforcement learning are very interesting from the viewpoint of biological evolution. This paper describes the effects of the interaction between the robot kinematics and the environment as a direct result of changing the environment.In addition, this study challenged the acquisition of two-dimensional motions with a starfish robot having four actuators. The result demonstrates that the robot can obtain a reasonable motion from the complicated relationships with the environment by skillfully employing its structure. Moreover, this paper implies that the reward manipulation may give a new insight for the learning process by the investigations performed in this study. This paper examines the possibility of the reward combinations for generating arbitrary motions.

Caterpillar robot locomotion based on Q-Learning using objective/subjective reward

“…In addition, the discount factor significantly affected the form of this maximum reward group. In [15] and [16], we stated that it is easy to produce the final motion form with very few motion patterns when the discount factor, , is set to a large value. In contrast, an inverse effect was confirmed when was set to a smaller value.…”

“…In our previous work, we studied the forward motion of a caterpillar robot using Q-learning, which is a typical and simple method of reinforcement learning [15], [16]. The results showed that reinforcement learning enabled the robot to achieve an unexpected motion pattern and exhibit good performance for a given task.…”

Research on Vibration Reduction Control Based on Reinforcement Learning

“…Littman and Michael et al [22] used Q-learning algorithm to realize autonomous movement control of robots. Hara et al [23] used machine learning control algorithms to control robots and improve learning efficiency.…”