Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach

Huang, Xuefei; Hong, Seung Ho; Yu, Mengmeng; Ding, Yuemin; Jiang, Junhui

doi:10.1109/access.2019.2924030

Cited by 56 publications

(18 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to conventional methods (FIFO, random forest), the deep RL algorithm was able to increase profits and minimise carbon consumption, while optimising lead time and cost. Considering a steel powder manufacturing process, Huang et al (2019) proposed a model-free control design to optimise the energy consumption plan based on current energy costs and individual process components (i.e. atomizer, crusher).…”

Section: Energy Managementmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

126

View full text Add to dashboard Cite

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

show abstract

Section: Energy Managementmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

126

View full text Add to dashboard Cite

show abstract

“…A concise review of DRL-based smart building energy management methods is summarized in [21]. More recently, various DRL-based BEMS methods have been developed, including the scheduling of the ESS and HVAC in residential buildings based on a deep deterministic policy gradient (DDPG) method [22], the management of utility-scale interruptible loads in a dueling deep Q network [23], actor-critic-based demand response management for industrial facilities [24], and the control of the state of charge of a group of multiple ESSs using the DDPG method [25].…”

Section: Introductionmentioning

confidence: 99%

Privacy-Preserving Energy Management of a Shared Energy Storage System for Smart Buildings: A Federated Deep Reinforcement Learning Approach

Lee

Xie

Choi

2021

Sensors

View full text Add to dashboard Cite

This paper proposes a privacy-preserving energy management of a shared energy storage system (SESS) for multiple smart buildings using federated reinforcement learning (FRL). To preserve the privacy of energy scheduling of buildings connected to the SESS, we present a distributed deep reinforcement learning (DRL) framework using the FRL method, which consists of a global server (GS) and local building energy management systems (LBEMSs). In the framework, the LBEMS DRL agents share only a randomly selected part of their trained neural network for energy consumption models with the GS without consumer’s energy consumption data. Using the shared models, the GS executes two processes: (i) construction and broadcast of a global model of energy consumption to the LBEMS agents for retraining their local models and (ii) training of the SESS DRL agent’s energy charging and discharging from and to the utility and buildings. Simulation studies are conducted using one SESS and three smart buildings with solar photovoltaic systems. The results demonstrate that the proposed approach can schedule the charging and discharging of the SESS and an optimal energy consumption of heating, ventilation, and air conditioning systems in smart buildings under heterogeneous building environments while preserving the privacy of buildings’ energy consumption.

show abstract

“…Xia et al proposed a digital twin approach for smart manufacturing, in which a deep Q-neural network (DQN) agent is trained in virtual systems to establish an optimal policy and can drive decision makings for operation in a real-world system [17]. Huang et al proposed an RL-based demand response (DR) scheme for steel power manufacturing, where actorcritic-based deep reinforcement learning (DRL) is utilized for efficient scheduling [18]. The agent reduced energy costs of the manufacturing process with efficient manufacturing schedule through DR. RL-based scheduling frameworks have been applied to not only manufacturing processes but also various plants, such as vinyl acetate monomer plant, circulating fluidized bed plant, coal-fired power plant, nuclear power plant and WWTP [19]- [24].…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning-Based Smart Joint Control Scheme for On/Off Pumping Systems in Wastewater Treatment Plants

et al. 2021

View full text Add to dashboard Cite

In this paper, we propose a deep reinforcement learning (DRL) based predictive control scheme for reducing the energy consumption and energy cost of pumping systems in wastewater treatment plants (WWTP), in which the pumps are operated in a binary mode, using on/off signals. As global energy consumption increases, the efficient operation of energy-intensive facilities has also become important. A WWTP in Busan, Republic of Korea is used as the target of this study. This WWTP is a large energyconsuming facility, and the pumping station accounts for a significant portion of the energy consumption of the WWTP. The framework of the proposed scheme consists of a deep neural network (DNN) model for forecasting wastewater inflow and a DRL agent for controlling the on/off signals of the pumping system, where proximal policy optimization (PPO) and deep Q-neural network (DQN) are employed as the DRL agents. To implement smart control with DRL, a reward function is designed to consider the energy consumption amount and electricity price information. In particular, new features and penalty factors for pump switching, which are essential for preventing pump wear, are also considered. The performance of our designed DRL agents is compared with those of WWTP experts and conventional approaches such as scheduling method and model predictive control (MPC), in which integer linear programming (ILP) optimization is employed. Results show that the designed agents outperform the other approaches in terms of compliance with operating rules and reducing energy costs.

show abstract

Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach

Cited by 56 publications

References 40 publications

Deep reinforcement learning in production systems: a systematic literature review

Deep reinforcement learning in production systems: a systematic literature review

Privacy-Preserving Energy Management of a Shared Energy Storage System for Smart Buildings: A Federated Deep Reinforcement Learning Approach

Deep Reinforcement Learning-Based Smart Joint Control Scheme for On/Off Pumping Systems in Wastewater Treatment Plants

Contact Info

Product

Resources

About