Development of an MCTS Model for Hydrogen Production Optimisation

Abstract

Hydrogen has the potential to revolutionize the energy industry due to its clean-burning and versatile properties. It is the most abundant element in the universe and can be produced through a variety of methods, including electrolysis. The widespread adoption of hydrogen faces various challenges, including the high cost of production; thus, it is important to optimise the production processes. This research focuses on development of models for hydrogen production optimisation based on various external factors and parameters. Models based on electricity prices are developed and compared between different market situations. To run hydrogen production more effectively, it is required to use renewable energy sources for the production process. Adding the solar power component to the economic evaluation model outcome is more positive. The Monte Carlo tree search (MCTS) algorithm is adapted to effectively control the electrolysis process. MCTS schedule optimization was performed for a 24 h time horizon applying two time-resolution settings—1 h and 15 min. The results demonstrate the potential of the MCTS algorithm for finding good schedules for water electrolyser devices by taking into account variable environmental factors. Whereas the MCTS with a 15 min resolution ensures mathematically better results, it requires more computational power to solve the decision tree.

Model for Hydrogen Production Scheduling Optimisation

“…where G + denotes sub-graph of states accessible from the previous state. We employed a simplified hydrogen production model described in detail in [14]. The model has three discrete production states, expressed as low, medium and high.…”

“…Table 1 below summarises all the variables used in research and calculations: The following components that are used in the model are the same as described in our previous work [14]:…”

“…Similar to the MCTS model, the CP-SAT model was optimised for a 24 h schedule with 1 h resolution and starting time t0 at midnight with the electrolyser being switched off (thus the first valid states are either continued Off state or the beginning of StartUp sequence). Figure 1 shows the comparison between a manually created "full power" schedule, the results from a previously developed [14] MCTS algorithm and the best solution found by the CP-SAT model. The model outputs are as follows: total energy consumption E, hydrogen produced QH2, oxygen produced QO2 and profit F. Despite almost doubling energy consumption, the CP-SAT model found slightly better cumulative profit for a given energy price profile.…”

“…Previously, we developed an MCTS (Monte Carlo Tree Search) model for hydrogen production optimisation [14], but it was found that the model is not able to find a truly optimal solution in very deep trees and thus it is necessary to search for other approaches for solving this problem. While MCTS has proven to be successful in various applications, it also comes with certain limitations and challenges, such as limited exploration in early stages and it may struggle to explore the search space effectively in the early stages of the search [15].…”

“…The efficiency of hydrogen electrolysers can be greatly improved by applying appropriate control strategies [18]. The implementation of such strategies is crucial because various parameters, like the frequency of electrolyser on/off cycles, influence the overall system performance [14].…”