An intelligent agent for optimal river‐reservoir system management

Rieker, Jeffrey D.; Labadie, John W.

doi:10.1029/2012wr011958

Cited by 21 publications

(15 citation statements)

References 33 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…where the solution is constrained by and must adhere to quality targets [17,42,91]). The mathematical programming techniques applied in this research include linear programming [80,103,178,197], nonlinear programming [17,91,121,150,155,206], dynamic programming [138,174,241], stochastic programming [33,42,70,99,128,139,141,187,207,208,224,225,240], and quadratic programming [86,113].…”

Section: Water Qualitymentioning

confidence: 99%

“…Marinoni et al [140] propose a framework for planning major investment decisions and apply this to the case of a water quality enhancement program in a river catchment in Brisbane, Australia. Compromise programming is first used to score the options [42,87,118,121,126,132,139,150,174,176,208,213,224] Storm water [17] Wastewater [1,33,65,80,86,98,106,113,129,139,141,169,187,197,206,219,238] Water allocation [172,230,233] Water trading [224] Water treatment [33,206] Wetlands [39,207,225] for pollution reduction at various sites and the optimal investment problem is then formulated as a multicriteria knapsack problem. In some cases, legislation needs to be considered alongside other management strategies.…”

Section: Water Qualitymentioning

confidence: 99%

See 1 more Smart Citation

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Archibald

Marshall

2018

Environ Model Assess

View full text Add to dashboard Cite

Economic development, variation in weather patterns and natural disasters focus attention on the management of water resources. This paper reviews the literature on the development of mathematical programming models for water resource management under uncertainty between 2010 and 2017. A systematic search of the academic literature identified 448 journal articles on water resource management for examination. Bibliometric analysis is employed to investigate the methods that researchers are currently using to address this problem and to identify recent trends in research in the area. The research reveals that stochastic dynamic programming and multistage stochastic programming are the methods most commonly applied. Water resource allocation, climate change, water quality and agricultural irrigation are amongst the most frequently discussed topics in the literature. A more detailed examination of the literature on each of these topics is included. The findings suggest that there is a need for mathematical programming models of large-scale water systems that deal with uncertainty and multiobjectives in an effective and computationally efficient way.

show abstract

Section: Water Qualitymentioning

confidence: 99%

Section: Water Qualitymentioning

confidence: 99%

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Archibald

Marshall

2018

Environ Model Assess

View full text Add to dashboard Cite

show abstract

“…Rather, the optimization problem is framed as a learning process where an agent (the decision maker) iteratively learn the optimal operating policy by taking actions (decisions) and experimenting the associated costs/rewards from the environment (the system). Single objective Reinforcement Learning has been extensively studied and experimented in many engineering applications, with several studies reported also in the water resources literature [ Lee and Labadie , ; Castelletti et al ., ; Rieker and Labadie , ]. On the other hand, multiobjective Reinforcement Learning (MORL) is still a relatively unexplored research area and, to the authors' knowledge, there have been no applications of MORL to water resources management.…”

Section: Introductionmentioning

confidence: 99%

A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run

2013

View full text Add to dashboard Cite

[1] The operation of large-scale water resources systems often involves several conflicting and noncommensurable objectives. The full characterization of tradeoffs among them is a necessary step to inform and support decisions in the absence of a unique optimal solution. In this context, the common approach is to consider many single objective problems, resulting from different combinations of the original problem objectives, each one solved using standard optimization methods based on mathematical programming. This scalarization process is computationally very demanding as it requires one optimization run for each trade-off and often results in very sparse and poorly informative representations of the Pareto frontier. More recently, bio-inspired methods have been applied to compute an approximation of the Pareto frontier in one single run. These methods allow to acceptably cover the full extent of the Pareto frontier with a reasonable computational effort. Yet, the quality of the policy obtained might be strongly dependent on the algorithm tuning and preconditioning. In this paper we propose a novel multiobjective Reinforcement Learning algorithm that combines the advantages of the above two approaches and alleviates some of their drawbacks. The proposed algorithm is an extension of fitted Q-iteration (FQI) that enables to learn the operating policies for all the linear combinations of preferences (weights) assigned to the objectives in a single training process. The key idea of multiobjective FQI (MOFQI) is to enlarge the continuous approximation of the value function, that is performed by single objective FQI over the state-decision space, also to the weight space. The approach is demonstrated on a real-world case study concerning the optimal operation of the HoaBinh reservoir on the Da river, Vietnam. MOFQI is compared with the reiterated use of FQI and a multiobjective parameterization-simulationoptimization (MOPSO) approach. Results show that MOFQI provides a continuous approximation of the Pareto front with comparable accuracy as the reiterated use of FQI. MOFQI outperforms MOPSO when no a priori knowledge on the operating policy shape is available, while produces slightly less accurate solutions when MOPSO can exploit such knowledge.Citation: Castelletti, A., F. Pianosi, and M. Restelli (2013), A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run, Water Resour. Res., 49,[3476][3477][3478][3479][3480][3481][3482][3483][3484][3485][3486]

show abstract

“…The results indicated the superiority of Q-learning over the other two approaches. Other examples of Q-learning applied for water resources systems can be found in Bhattacharya et al (2003), Mariano-Romero et al (2007), Mahootchi et al (2007), Mahootchi et al (2010), and Rieker and Labadie (2012). Castelletti et al (2010) applied the fitted Q-iteration combined with tree-based regression to form a suitable function approximator in daily operation of a single reservoir system in Italy.…”

mentioning

confidence: 99%

Comparative analysis of evolving artificial neural network and reinforcement learning in stochastic optimization of multireservoir systems

Dariane

Moradi

2016

Hydrological Sciences Journal

View full text Add to dashboard Cite

Dynamic Programming (DP) has been among the most popular techniques for solving multi-reservoir problems since early 1960s. However, DP and DP based methods suffer from two serious issues of namely the curses of modeling and dimensionality. Later, Reinforcement Learning (RL) was introduced to overcome some deficiencies in the traditional DP mainly related to the curse of modeling, but it still encounters the curse of dimensionality in larger systems. Recently, artificial neural network has emerged as an effective approach to solve stochastic optimization of reservoir systems with high flexibility. In this paper, we develop a single-step evolving artificial neural network (SENN) model that overcomes the curses of modeling and dimensionality in a multireservoir system. Furthermore, a novel efficient allocation technique is developed to ease the allocation of water among different users. A system of two-reservoir in Karkheh Basin, Iran, is applied to derive and test the methods. Elitist-mutated particle swarm optimization is used to train the network. A comparison of the results with the Q-learning shows the superiority of the SENN, especially during drought periods. Moreover, the SENN performs better in producing more hydropower energy in the system. Thus, main contributions of this research are: 1) development of SENN applications to multireservoir systems, 2) a comparative analysis between SENN and Q-learning especially in prolonged drought conditions and, 3) a proposed efficient optimal allocation technique using simulation method.

show abstract

An intelligent agent for optimal river‐reservoir system management

Cited by 21 publications

References 33 publications

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

Review of Mathematical Programming Applications in Water Resource Management Under Uncertainty

A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run

Comparative analysis of evolving artificial neural network and reinforcement learning in stochastic optimization of multireservoir systems

Contact Info

Product

Resources

About