No abstract

No abstract

Battery energy storage systems (BESSs) play a critical role in eliminating uncertainties associated with renewable energy generation, to maintain stability and improve flexibility of power networks. In this paper, a BESS is used to provide energy arbitrage (EA) and frequency regulation (FR) services simultaneously to maximize its total revenue within the physical constraints. The EA and FR actions are taken at different timescales. The multitimescale problem is formulated as two nested Markov decision process (MDP) submodels. The problem is a complex decision-making problem with enormous high-dimensional data and uncertainty (e.g., the price of the electricity). Therefore, a novel co-optimization scheme is proposed to handle the multitimescale problem, and also coordinate EA and FR services. A triplet deep deterministic policy gradient with exploration noise decay (TDD–ND) approach is used to obtain the optimal policy at each timescale. Simulations are conducted with real-time electricity prices and regulation signals data from the American PJM regulation market. The simulation results show that the proposed approach performs better than other studied policies in literature.

Multienergy microgrids (MEMGs) have significant potential to offer high energy utilization efficiency and system flexibility. The coordination of these MEMGs poses challenges due to the various system dynamics and uncertainties and the need to preserve privacy. This article proposes a double auction (DA)-market-based coordination framework. As such, MEMGs can not only schedule their own energy components but also trade energy with others in the DA market. After that, we formulate this problem as Markov games and propose a multiagent reinforcement learning method by making use of the DA market public information to enhance the stability with privacy perseverance. Case studies involving a real-world scenario validate the superior performance of the proposed method in reducing both the energy costs and the carbon emissions.