Reinforcement Learning-Based Method to Exploit Vulnerabilities of False Data Injection Attack Detectors in Modular Multilevel Converters

Abstract

Implementing control schemes for modular multilevel converters (M2Cs) involves both a cyber and a physical level, leading to a cyber-physical system (CPS). At the cyber level, a communication network enables the data exchange between sensors, control platforms, and monitoring systems. Meanwhile, at the physical level, the semiconductor devices that comprise the M2C are switched ON/OFF by the control system. In this context, almost all published works in this research area assume that the CPS always reports correct information. However, this may not be the case when the M2C is affected by cyber-attacks, such as the one named false data injection attack (FDIA), where the data seen by the control system is corrupted through illegitimate data intrusion into the CPS. To deal with this situation, FDIA detectors for the M2C are recently starting to be studied, where the goal is to detect and mitigate the attacks and the attacked sub-modules. This paper proposes a reinforcement learning (RL)-based method to uncover the deficiencies of existing FDIAs detectors used for M2C applications. The proposed method auto-generates complex attack sequences able to bypass FDIA detectors. Therefore, it points out the weaknesses of current detectors: This valuable information can be used later to improve the performance of the detectors, establishing more reliable cybersecurity solutions for M2Cs. The RL environment is developed in Matlab/Simulink augmented by PLECS/blockset, and it is made available to researchers on a website to motivate future research efforts in this area. Hardware-in-the-loop (HIL) studies verify the proposal's effectiveness.

A Review of Cybersecurity in Grid-Connected Power Electronics Converters: Vulnerabilities, Countermeasures, and Testbeds

“…Recently, some efforts were conducted to exploit vulnerabilities of established countermeasures of cybersecurity for GCPEC. Paper [45] of 2023 proposed a reinforcement learning (RL)-based method to uncover the deficiencies of existing false data injection attack (FDIA) detectors used for modular multilevel converters (M2C) applications, a prominent solution for high-efficient longdistance high-voltage direct current (HVdc) transmission systems. Depending on the defined RL scheme, it is necessary define the following elements to use the RL technique for obtaining the FDIA attacker: 1) the inputs of the actor, which is the neural network (NN) that will define the attack, and the critic, another NN that evaluates the cost; 2) the output of the actor; 3) the reward function that drives the training; 4) the experiment design [45].…”

“…Paper [45] of 2023 proposed a reinforcement learning (RL)-based method to uncover the deficiencies of existing false data injection attack (FDIA) detectors used for modular multilevel converters (M2C) applications, a prominent solution for high-efficient longdistance high-voltage direct current (HVdc) transmission systems. Depending on the defined RL scheme, it is necessary define the following elements to use the RL technique for obtaining the FDIA attacker: 1) the inputs of the actor, which is the neural network (NN) that will define the attack, and the critic, another NN that evaluates the cost; 2) the output of the actor; 3) the reward function that drives the training; 4) the experiment design [45]. The effectiveness of this RL method was verified in HIL studies, which found that the attack sequences depend on the characteristics of the FDIA detector studies, i.e., the more sophisticated the FDIA detectors, the more complex attack sequences will be generated by the proposed RL-method.…”

Detection of Stealthy False Data Injection Attacks in Modular Multilevel Converters

“…In follow-up studies, ref. [27] presents a reinforcement learning-based method to exploit the vulnerabilities of FDIA detectors in the MMC. The proposed method reveals the weakness of the fault detector given in [26] and provides a solution for researchers to motivate future research in this area.…”

“…To the best of our knowledge, refs. [25][26][27] are all the related publications regarding the topic of cyber-attack in the MMC.…”

“…Compared to the FDIA step analyzed in [26], stealthy false data injection with a more elaborate attack sequence can deceive the protection system and bypass the fault detector [27]. This type of FDIA is more dangerous to the safe operation of CPS.…”