Deep reinforcement learning for the real time control of stormwater systems

Mullapudi, Abhiram; Lewis, Matthew; Gruden, Cyndee; Kerkez, Branko

doi:10.1016/j.advwatres.2020.103600

Cited by 92 publications

(50 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Operation: Real-time Operation Controlmentioning

confidence: 99%

“…Several researchers [32,33] have proposed methodologies for real-time operation control in the UDS domain. Mullapudi et al (2020) [32] formulated and analyzed a real-time operation control model (i.e., pumps) using reinforcement learning. Consequently, the automated operation system for UDSs demonstrated that reinforcement learning can effectively and efficiently pump operation and control for individual subcatchments.…”

Section: Operation: Real-time Operation Controlmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning and Urban Drainage Systems: State-of-the-Art Review

Kwon

Kim

2021

Water

View full text Add to dashboard Cite

In the last decade, machine learning (ML) technology has been transforming daily lives, industries, and various scientific/engineering disciplines. In particular, ML technology has resulted in significant progress in neural network models; these enable the automatic computation of problem-relevant features and rapid capture of highly complex data distributions. We believe that ML approaches can address several significant new and/or old challenges in urban drainage systems (UDSs). This review paper provides a state-of-the-art review of ML-based UDS modeling/application based on three categories: (1) operation (real-time operation control), (2) management (flood-inundation prediction) and (3) maintenance (pipe defect detection). The review reveals that ML is utilized extensively in UDSs to advance model performance and efficiency, extract complex data distribution patterns, and obtain scientific/engineering insights. Additionally, some potential issues and future directions are recommended for three research topics defined in this study to extend UDS modeling/applications based on ML technology. Furthermore, it is suggested that ML technology can promote developments in UDSs. The new paradigm of ML-based UDS modeling/applications summarized here is in its early stages and should be considered in future studies.

show abstract

Section: Operation: Real-time Operation Controlmentioning

confidence: 99%

Section: Operation: Real-time Operation Controlmentioning

confidence: 99%

Machine Learning and Urban Drainage Systems: State-of-the-Art Review

Kwon

Kim

2021

Water

View full text Add to dashboard Cite

show abstract

“…In this method, an agent is proposed that relies over the dueling Deep Q-network concept (a variant of DQN) to control the pump operation. In [125], a control strategy is proposed as a real-time control strategy for the stormwater management, using DRL. Readers can refer to [142] for additional details of DRL and Deep Q-Networks.…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 99%

Cyber-Physical Systems for Smart Water Networks: A Review

et al. 2021

View full text Add to dashboard Cite

There is a growing demand to equip Smart Water Networks (SWN) with advanced sensing and computation capabilities in order to detect anomalies and apply autonomous event-triggered control. Cyber-Physical Systems (CPSs) have emerged as an important research area capable of intelligently sensing the state of SWN and reacting autonomously in scenarios of unexpected crisis development. Through computational algorithms, CPSs can integrate physical components of SWN, such as sensors and actuators, and provide technological frameworks for data analytics, pertinent decision making, and control. The development of CPSs in SWN requires the collaboration of diverse scientific disciplines such as civil, hydraulics, electronics, environment, computer science, optimization, communication, and control theory. For efficient and successful deployment of CPS in SWN, there is a need for a common methodology in terms of design approaches that can involve various scientific disciplines. This paper reviews the state of the art, challenges, and opportunities for CPSs, that could be explored to design the intelligent sensing, communication, and control capabilities of CPS for SWN. In addition, we look at the challenges and solutions in developing a computational framework from the perspectives of machine learning, optimization, and control theory for SWN.

show abstract

“…Hsu et al [31] used an adaptive network-based fuzzy information system to build two real-time pumping station operation models based on the historical operation records (ANFIS-His) and the best operation series (ANFIS-Opt), and the result showed ANFIS-Opt was better than ANFIS-His for the drainage system in New Taipei city. Mullapudi et al [32] formulated a reinforcement learning (RL) algorithm for the real-time control of urban stormwater systems, where the results indicated that RL could effectively control individual sites in an urban watershed in Ann Arbor. Pereira et al [33] designed a PID controller to automate the operation of the Lordelo pumping station based on the intake water level measurements.…”

Section: Introductionmentioning

confidence: 99%

A Simple Method for the Control Time of a Pumping Station to Ensure a Stable Water Level Immediately Upstream of the Pumping Station under a Change of the Discharge in an Open Channel

Yan

Zhang

Lei

et al. 2021

Water

View full text Add to dashboard Cite

For an open channel with cascade pumping stations, the water level immediately upstream of the pumping station should be kept constant to ensure pumping stability and lining safety. In this study, a simple method was proposed to determine the control time of the pumping station to ensure a stable water level immediately upstream of the pumping station using reverse analysis. The variable discharge process and fixed water level were taken as the upstream and downstream boundaries of the one-dimensional open channel hydrodynamic model, and the discharge process of the pumping station was obtained under ideal conditions. The control time was determined by the equivalent water volume change considering the step change of the discharge of the pumping station. A case study was performed using the Jiaodong water diversion project from Songzhuang sluice to Huibu pumping station (G1–P1), and the effects of the initial discharge, variable discharge, and downstream water level on the control time were investigated. The results show that (1) the larger the initial discharge, the shorter the control time; (2) increasing the upstream discharge reduced the control time, while decreasing the upstream discharge increased the control time; (3) the control time decreased with the increase of the water depth immediately upstream of the pumping station.

show abstract

Deep reinforcement learning for the real time control of stormwater systems

Cited by 92 publications

References 21 publications

Machine Learning and Urban Drainage Systems: State-of-the-Art Review

Machine Learning and Urban Drainage Systems: State-of-the-Art Review

Cyber-Physical Systems for Smart Water Networks: A Review

A Simple Method for the Control Time of a Pumping Station to Ensure a Stable Water Level Immediately Upstream of the Pumping Station under a Change of the Discharge in an Open Channel

Contact Info

Product

Resources

About