Preference based multi-objective reinforcement learning for multi-microgrid system optimization problem in smart grid

Xu, Jiangjiao; Li, Ké; Abusara, Mohammad

doi:10.1007/s12293-022-00357-w

Cited by 15 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As discussed in Section 3.1, the hyperparameters of the LMC surges to 1, 000 when t = 10 whereas the number of training data is way smaller. In contrast, the HMOGP performs consistently well even when t = 10 as its number of hyperparameters is merely 10, which is manageable [97][98][99].…”

Section: Resultsmentioning

confidence: 93%

A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments

Li¹,

Chen²,

Yao³

2022

Preprint

View full text Add to dashboard Cite

Many real-world problems are usually computationally costly and the objective functions evolve over time. Data-driven, a.k.a. surrogate-assisted, evolutionary optimization has been recognized as an effective approach for tackling expensive black-box optimization problems in a static environment whereas it has rarely been studied under dynamic environments. This paper proposes a simple but effective transfer learning framework to empower data-driven evolutionary optimization to solve dynamic optimization problems. Specifically, it applies a hierarchical multi-output Gaussian process to capture the correlation between data collected from different time steps with a linearly increased number of hyperparameters. Furthermore, an adaptive source task selection along with a bespoke warm staring initialization mechanisms are proposed to better leverage the knowledge extracted from previous optimization exercises. By doing so, the data-driven evolutionary optimization can jump start the optimization in the new environment with a strictly limited computational budget. Experiments on synthetic benchmark test problems and a real-world case study demonstrate the effectiveness of our proposed algorithm against nine state-of-the-art peer algorithms.

show abstract

Section: Resultsmentioning

confidence: 93%

A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments

Li¹,

Chen²,

Yao³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In addition, we can consider more complicated scenarios with multiple non-functional objectives and challenging constraints [75]. In addition, we can also take the decision makers' preference information [76][77][78][79][80][81][82][83][84] into the evolutionary search to generate more personalized examples [85].…”

Section: Discussionmentioning

confidence: 99%

Evolutionary Multi-Task Injection Testing on Web Application Firewalls

Li¹,

Heng²,

Visser³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Web application firewall (WAF) plays an integral role nowadays to protect web applications from various malicious injection attacks such as SQL injection, XML injection, and PHP injection, to name a few. However, given the evolving sophistication of injection attacks and the increasing complexity of tuning a WAF, it is challenging to ensure that the WAF is free of injection vulnerabilities such that it will block all malicious injection attacks without wrongly affecting the legitimate message. Automatically testing the WAF is, therefore, a timely and essential task. In this paper, we propose DaNuoYi, an automatic injection testing tool that simultaneously generates test inputs for multiple types of injection attacks on a WAF. Our basic idea derives from the cross-lingual translation in the natural language processing domain. In particular, test inputs for different types of injection attacks are syntactically different but may be semantically similar. Sharing semantic knowledge across multiple programming languages can thus stimulate the generation of more sophisticated test inputs and discovering injection vulnerabilities of the WAF that are otherwise difficult to find. To this end, in DaNuoYi, we train several injection translation models by using multi-task learning that translates the test inputs between any pair of injection attacks. The model is then used by a novel multi-task evolutionary algorithm to co-evolve test inputs for different types of injection attacks facilitated by a shared mating pool and domain-specific mutation operators at each generation. We conduct experiments on three real-world open-source WAFs and six types of injection attacks, the results reveal that DaNuoYi generates up to 3.8× and 5.78× more valid test inputs (i.e., bypassing the underlying WAF) than its state-of-the-art single-task counterparts and the context-free grammar-based injection construction.

show abstract

“…Things become even more challenging when the constraints are (partially) unobservable [80,81]. It is also worth noting that evolutionary computation and multi-objective optimization have been successfully applied to solve real-world problems, e.g., natural language processing [82], neural architecture search [83][84][85][86], robustness of neural networks [87][88][89][90][91][92], software engineering [93][94][95][96][97], smart grid management [2,98,99], communication networks [100][101][102][103], machine learning [104][105][106][107][108], and visualization [109]. Finally, the explainability of the policies of interest and their implications has rarely been discussed in the literature, representing another area for future investigation.…”

Section: Discussionmentioning

confidence: 99%

An Improved Multi-objective Optimization Algorithm Based on Reinforcement Learning

Liu

Zhou

Qiu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Multi-objective reinforcement learning (MORL) aims to find a set of high-performing and diverse policies that address trade-offs between multiple conflicting objectives. However, in practice, decision makers (DMs) often deploy only one or a limited number of trade-off policies. Providing too many diversified trade-off policies to the DM not only significantly increases their workload but also introduces noise in multi-criterion decision-making. With this in mind, we propose a humanin-the-loop policy optimization framework for preference-based MORL that interactively identifies policies of interest. Our method proactively learns the DM's implicit preference information without requiring any a priori knowledge, which is often unavailable in real-world black-box decision scenarios. The learned preference information is used to progressively guide policy optimization towards policies of interest. We evaluate our approach against three conventional MORL algorithms that do not consider preference information and four state-of-the-art preference-based MORL algorithms on two MORL environments for robot control and smart grid management. Experimental results fully demonstrate the effectiveness of our proposed method in comparison to the other peer algorithms.

show abstract

Preference based multi-objective reinforcement learning for multi-microgrid system optimization problem in smart grid

Cited by 15 publications

References 26 publications

A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments

A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments

Evolutionary Multi-Task Injection Testing on Web Application Firewalls

An Improved Multi-objective Optimization Algorithm Based on Reinforcement Learning

Contact Info

Product

Resources

About