Deep Reinforcement Learning Based Self-Configuring Integral Sliding Mode Control Scheme for Robot Manipulators

Abstract

This paper deals with the design of an intelligent self-configuring control scheme for robot manipulators. The scheme features two control structures: one of centralized type, implementing the inverse dynamics approach, the other of decentralized type. In both control structures, the controller is based on Integral Sliding Mode (ISM), so that matched disturbances and uncertain terms, due to unmodeled dynamics or couplings effects, are suitably compensated. The use of the ISM control also enables the exploitation of its capability of acting as a "perturbation estimator" which, in the considered case, allows us to design a Deep Reinforcement Learning (DRL) based decision making mechanism. It implements a switching rule, based on an appropriate reward function, in order to choose one of the two control structures present in the scheme, depending on the requested robot performances. The proposed scheme can accommodate a variety of velocity and acceleration requirements, in contrast with the genuine decentralized or centralized control structures taken individually. The assessment of our proposal has been carried out relying on a model of the industrial robot manipulator COMAU SMART3-S2, identified on the basis of real data and with realistic sensor noise.

Networked Control of Nonlinear Systems under Partial Observation Using Continuous Deep Q-Learning

“…In [15], DRL is applied to a parking problem of a 4-wheeled vehicle that is a nonholonomic system. In [16], DRL is applied to controlling robot manipulators. Moreover, applications of DRL to networked control systems (NCSs) have also been proposed [17], [18], [20].…”