Space shuttle attitude control by reinforcement learning and fuzzy logic

Abstract

In this paper, we discuss the results of applying the ARIC and GARIC architectures, which have been developed for reinforcement learning using fuzzy logic, to the attitude control of the Space Shuttle. This paper demonstrates that it is possible to control the pitch, roll, and yaw of the Space Shuttle within a specified deadband by using fuzzy control rules and automatically adapt to a reduced error tolerance. The performance of this controller is compared with a controller using conventional control theory and also a non-adaptive fuzzy controller. Our results, using the Orbital Operations Simulator (00s) system, demonstrate that more difficult tasks can be learned by our controller while the fuel efficiency remains very high.

Adaptive fuzzy control of satellite attitude by reinforcement learning

“…The sum of future reinforcements which the critic learns to predict is given by (1) where is an exponential discounting factor, is the external reinforcement signal, denotes a discrete time instant, and is the discounted sum of future reinforcements (also called the value function). The critic is usually implemented as a nonlinear function approximator such as a neural network [7] or a fuzzy system [4]. In order to derive an update law for the critic parameters, let us denote the prediction of computed by the critic.…”

“…The Takagi-Sugeno (TS) rules with constant consequent functions are used in this paper to implement both the critic and the controller while the controller in GARIC is based on linguistic rules. The TS scheme is computationally more efficient, equally transparent to interpretation of both the initial and the adapted controller and provides intuitively more understandable and consistent results than those presented in [4]. • Since there is no failure situation defined, no backup controller is used.…”

“…These methods are usually referred to as neuro-fuzzy control (see, for instance, [2] among many other references). One of the neuro-fuzzy control techniques is based on a combination of fuzzy modeling and control structures with reinforcement learning (RL) [3], [4]. The main advantage of reinforcement learning is that no model of the process is required for the adaptation of the controller [5].…”

“…As the noise characteristics vary for different objects (weak and strong stars), adaptation features are desirable in order to maintain satisfactory performance under all conditions. Berenji et al [4] demonstrated that adaptive fuzzy control by means of RL can be applied to attitude control. Our approach is also based on reinforcement learning, but there are several important differences between the scheme of Berenji et al called GARIC and the approach presented in this paper.…”

“…• The update law for the controller parameters is based on the original approach of Barto [5] instead of using an approximation of the process Jacobian as in [4]. The rest of this paper is organized as follows.…”