Chaos synchronization of two coupled map lattice systems using safe reinforcement learning

Abstract

Compared to the synchronization of continuous-time chaotic systems which will usually satisfy the Lipschitz condition, rapid trajectory divergence is a key challenge in the synchronization of two high-dimensional discrete chaotic systems, for example two coupled map lattice systems. As a result, there is not yet a universal approach to the synchronization task in high-dimensional discrete chaotic systems. To overcome the challenge, hard constraints on the system states must be satisfied, which is defined as safety level III. We propose a safe reinforcement learning (RL) method with this high safety level. In this method, the RL agent’s policy is used to reach the goal of synchronization and a safety layer added directly on top of the policy is used to guarantee hard state constraints. The safety layer consists of a one-step predictor for the perturbed response system and an action correction formulation. The one-step predictor, based on a next generation reservoir computing, is used to identify whether the next state of the perturbed system is within the chaos domain, and if not, the action correction formula is activated to modify the corresponding perturbing force component to zero. According to the boundedness of chaotic systems, the state of the perturbed system will remain in the chaotic domain without diverging. We demonstrate that the proposed method succeeds in the task of synchronization without trajectory divergence through a numerical example with two coupled map lattice systems. We compare the performance in both cases with and without the safety layer to emphasize the significance of the safety layer and analyze the effect of hyper-parameters on the performance and stability of the algorithm.