Deep Q-Network Based Energy Scheduling in Retail Energy Market

Abstract

As a significant component of electric energy trades, retail electric market (REM) can effectively alleviate the pressure of load demand from the power grid. However, the load demand uncertainty of customers becomes a nodus because retail electricity providers (REPs) should predict the load demand when trading with wholesaler electricity provider (WEP) based on the interaction. Therefore, in this paper, we propose an optimal energy scheduling scheme in REM with consideration of the influence of decisions made in pre-purchase stages to situations in real-time stages. Firstly, we present a trading framework to analyze the strategies of REPs in REM, in which REPs conduct both pre-purchase trading with WEP, and real-time trading with customers. Then, to solve the scheduling problem caused by the demand uncertainty of customers, we design a power allocation mechanism based on the charging demand degree, by which REPs can minimize the operating cost while ensuring that each electric vehicle can be charged with sufficient energy. Next, to minimize the cost of REPs in the pre-purchase stage, we adopt deep Q-network (DQN) algorithm to implement the pre-purchase schedule. The charging station adjusts the pre-purchased schedule for each period through Q-learning and utilizes the optimal strategy to design the electricity schedule. Finally, simulation experiments show that the proposal can obtain the optimal strategy to significantly reduce the operating costs of REP. INDEX TERMS Retail electricity market (REM), plug-in electric vehicles (PEVs), deep Q-network (DQN). I. INTRODUCTION

Decoding topological XYZ 2 codes with reinforcement learning based on attention mechanisms

“…In reinforcement learning, an autonomous agent interacts with the environment through its own experience, [39] observes the state of the environment to take action, and learns its own behaviors with the help of feedback from the environment to maximize the cumulative rewards. These behaviors tend to become more optimal as the agent gains experience.…”

“…These behaviors tend to become more optimal as the agent gains experience. [39,40] This can be solved by employing a Markov decision process [41] consisting of five different entities: state, action, reward, policy, and value. Reinforcement learning can be represented by a quaternion ⟨S, A, P, R⟩, where S is the set of states, A is the set of actions, P is the state transfer function, and R is the reward function.…”

Approximate error correction scheme for three-dimensional surface codes based reinforcement learning

“…The agent operates on the environment and performs a series of actions to solve certain problems. [37,38] Each time step t can be represented by a state s t ∈ S, where S is the state space, showing how the environment is represented. The feedback loop between the agent and the environment is called a Markov decision process (MDP).…”