Comparative analysis of evolving artificial neural network and reinforcement learning in stochastic optimization of multireservoir systems

Dariane, Alireza B.; Moradi, A.

doi:10.1080/02626667.2014.986485

Cited by 17 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL has two essential characteristics: trial‐and‐error search and delayed reward, which enable agents to adapt their strategies by interacting with the environment. Since the introduction of RL to the water resources field, it has been extensively applied for optimal reservoir operations (Castelletti et al., 2010; Dariane & Moradi, 2016; Lee & Labadie, 2007; Madani & Hooshyar, 2014; Rieker & Labadie, 2012), with a few exceptions in dam sizing (Bertoni et al., 2020) and water and natural resource allocations (Bone & Dragićević, 2009; Ni et al., 2014). However, despite its increasing popularity, these applications only focus on RL's learning aspect for finding optimal policies in a stationary environment.…”

Section: Introductionmentioning

confidence: 99%

Assessing Adaptive Irrigation Impacts on Water Scarcity in Nonstationary Environments—A Multi‐Agent Reinforcement Learning Approach

Hung

Yang

2021

Water Resources Research

View full text Add to dashboard Cite

One major challenge in water resource management is to balance the uncertain and nonstationary water demands and supplies caused by the changing anthropogenic and hydroclimate conditions. To address this issue, we developed a reinforcement learning agent‐based modeling (RL‐ABM) framework where agents (agriculture water users) are able to learn and adjust water demands based on their interactions with the water systems. The intelligent agents are created by a reinforcement learning algorithm adapted from the Q‐learning algorithm. We illustrated this framework in a case study where the RL‐ABM is two‐way coupled with the Colorado River Simulation System (CRSS), a long‐term planning model used for the administration of the Colorado River Basin, for assessing agriculture water uses impacts on water scarcity. Seventy‐eight intelligent agents are simulated, which can be grouped into three categories based on their parameter values: the “aggressive” (swift actions; low regrets), the “forward‐looking conservative” (mild actions; high regrets; fast learning), and the “myopic conservative” (mild actions; median regrets; slow learning). The ABM‐CRSS results showed that the major reservoirs in the Upper Colorado Basin might experience more frequent water shortages due to the increasing water uses compared to the original CRSS results. If the drought continues, the case study also demonstrates that agents can learn and adjust their demands.

show abstract

Section: Introductionmentioning

confidence: 99%

Assessing Adaptive Irrigation Impacts on Water Scarcity in Nonstationary Environments—A Multi‐Agent Reinforcement Learning Approach

Hung

Yang

2021

Water Resources Research

View full text Add to dashboard Cite

show abstract

“…Ref. [27] uses ML to overcome the problem in deriving complex models as occurs in multipurpose multi-reservoir systems. Therefore, ANN is applied to derive the optimized reservoir release, solving a multi-objective function: minimize water demand deficits and reservoir spills as convex functions while maximizing hydropower energy production as a nonconvex function.…”

Section: Results Analysismentioning

confidence: 99%

Hydropower Operation Optimization Using Machine Learning: A Systematic Review

Bernardes

Santos

Abreu

et al. 2022

View full text Add to dashboard Cite

The optimal dispatch of hydropower plants consists of the challenge of taking advantage of both available head and river flows. Despite the objective of delivering the maximum power to the grid, some variables are uncertain, dynamic, non-linear, and non-parametric. Nevertheless, some models may help hydropower generating players with computer science evolution, thus maximizing the hydropower plants’ power production. Over the years, several studies have explored Machine Learning (ML) techniques to optimize hydropower plants’ dispatch, being applied in the pre-operation, real-time and post-operation phases. Hence, this work consists of a systematic review to analyze how ML models are being used to optimize energy production from hydropower plants. The analysis focused on criteria that interfere with energy generation forecasts, operating policies, and performance evaluation. Our discussions aimed at ML techniques, schedule forecasts, river systems, and ML applications for hydropower optimization. The results showed that ML techniques have been more applied for river flow forecast and reservoir operation optimization. The long-term scheduling horizon is the most common application in the analyzed studies. Therefore, supervised learning was more applied as ML technique segment. Despite being a widely explored theme, new areas present opportunities for disruptive research, such as real-time schedule forecast, run-of-river system optimization and low-head hydropower plant operation.

show abstract