An important goal of China's electric power system reform is to create a double-side day-ahead wholesale electricity market in the future, where the suppliers (represented by GenCOs) and demanders (represented by DisCOs) compete simultaneously with each other in one market. Therefore, modeling and simulating the dynamic bidding process and the equilibrium in the double-side day-ahead electricity market scientifically is not only important to some developed countries, but also to China to provide a bidding decision-making tool to help GenCOs and DisCOs obtain more profits in market competition. Meanwhile, it can also provide an economic analysis tool to help government officials design the proper market mechanisms and policies. The traditional dynamic game model and table-based reinforcement learning algorithm have already been employed in the day-ahead electricity market modeling. However, those models are based on some assumptions, such as taking the probability distribution function of market clearing price (MCP) and each rival's bidding strategy as common knowledge (in dynamic game market models), and assuming the discrete state and action sets of every agent (in table-based reinforcement learning market models), which are no longer applicable in a realistic situation. In this paper, a modified reinforcement learning method, called gradient descent continuous Actor-Critic (GDCAC) algorithm was employed in the double-side day-ahead electricity market modeling and simulation. This algorithm can not only get rid of the abovementioned unrealistic assumptions, but also cope with the Markov decision-making process with continuous state and action sets just like the real electricity market. Meanwhile, the time complexity of our proposed model is only O(n). The simulation result of employing the proposed model in the double-side day-ahead electricity market shows the superiority of our approach in terms of participant's profit or social welfare compared with traditional reinforcement learning methods.

Abstract:In this paper, considering real time wind power uncertainties, the strategic behaviors of wind power producers adopting two different bidding modes in day-ahead electricity market is modeled and experimentally compared. These two different bidding modes only provide a wind power output plan and a bidding curve consisting of bidding price and power output, respectively. On the one hand, to significantly improve wind power accommodation, a robust market clearing model is employed for day-ahead market clearing implemented by an independent system operator. On the other hand, since the Least Squares Continuous Actor-Critic algorithm is demonstrated as an effective method in dealing with Markov decision-making problems with continuous state and action sets, we propose the Least Squares Continuous Actor-Critic-based approaches to model and simulate the dynamic bidding interaction processes of many wind power producers adopting two different bidding modes in the day-head electricity market under robust market clearing conditions, respectively. Simulations are implemented on the IEEE 30-bus test system with five strategic wind power producers, which verify the rationality of our proposed approaches. Moreover, the quantitative analysis and comparisons conducted in our simulations put forward some suggestions about leading wind power producers to reasonably bid in market and bidding mode selections.

As China’s electricity market is facing many problems, the research on power producer’s bidding behavior can promote the healthy and sustainable development of China’s electricity market. As a special commodity, the “electricity” possesses complicated production process. The instable market constraint condition, nonsymmetric information, and a lot of random factors make the producer’s bidding process more complex. Best-response dynamic is one of the classic dynamic mechanisms of the evolutionary game theory, which applies well in the repeated game and strategy evolution that happen among a few bounded rational players with a quick learning capability. The best-response dynamic mechanism is employed to study the power producer’s bidding behavior in this paper, the producer’s best-response dynamic model is constructed, and how the producers would engage in bidding is analyzed in detail. Taking two generating units in South China regional electricity market as the example, the producer’s bidding behavior by following the producer’s best-response dynamic model is verified. The relationships between the evolutionarily stable strategy (ESS) of power producer’s bidding and the market demand, and ceiling and floor price as well as biding frequency are discussed in detail.