A Risk-Sensitive Intelligent Control Algorithm for Servo Motor Based on Value Distribution

Abstract

With the development of artificial intelligence, reinforcement-learning-based intelligent control algorithms, which generally learn control strategies through trial and error, have received more attention in the automation equipment and manufacturing fields. Although they can intelligently adjust their control strategy without the need for human effort, the most relevant algorithms for servo motors only consider the overall performance, while ignoring the risks in special cases. Therefore, overcurrent problems are often triggered in the training process of the reinforcement learning agent. This can damage the motors’ service life and even burn it out directly. To solve this problem, in this study we propose a risk-sensitive intelligent control algorithm based on value distribution, which uses the quantile function to model the probability distribution of cumulative discount returns and employs the condition value at risk to measure the loss caused by overcurrent. The agent can accordingly learn a control strategy that is more sensitive to environmental restrictions and avoid the overcurrent problem. The performance is verified on three different servo motors with six control tasks, and the experimental results show that the proposed method can achieve fewer overcurrent occurrences than others in most cases.