Control of Air Free-Cooled Data Centers in Tropics via Deep Reinforcement Learning

Le, Duc Van; Liu, Yingbo; Wang, Rongrong; Tan, Rui; Wong, Y. W.; Wen, Yonggang

doi:10.1145/3360322.3360845

Cited by 23 publications

(17 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This suits the practical interest especially when a detailed model is not available. In recent years, some reinforcement learning algorithms such as Deep Q Network (Van Le et al, 2019), Trust Region Policy Optimisation (Moriyama et al, 2018), Deep Deterministic Policy Gradient (Chi et al, 2020;Li, Wen, Tao, & Guan, 2019) have been already studied and applied to the energy-saving and reliable control of the data centre cooling system and achieved good energy-saving performance (Duan et al, 2020;Kumar, Khatri, & Diván, 2020;Linder, Van Gilder, Zhang, & Barrett, 2019;Liu, Wong, Ye, & Ma, 2020;Thein, Myo, Parvin, & Gawanmeh, 2020;Yang, Wang, He, Sun, & Zhang, 2019). Despite some considerable advantages of model-free reinforcement learning algorithms as aforementioned, the implementation in practice still faces many challenges.…”

Section: Smart Data Centresmentioning

confidence: 99%

Control for smart systems: Challenges and trends in smart cities

Jia

Panetto

Macchi

et al. 2022

Annual Reviews in Control

View full text Add to dashboard Cite

Section: Smart Data Centresmentioning

confidence: 99%

Control for smart systems: Challenges and trends in smart cities

Jia

Panetto

Macchi

et al. 2022

Annual Reviews in Control

View full text Add to dashboard Cite

“…Therefore, in this work, we focus on compare our new cDRL approach with the uDRL and hysteresis-based approaches only. The detailed results of the MPC approach can be found in our preliminary work [38]. Fig.…”

Section: Drl Agent Executionmentioning

confidence: 99%

“…A preliminary version of this work [38] presented an unconstrained DRL-based approach for controlling the air free-cooled DCs with the main objective of minimizing the cooling energy consumption while satisfying the requirements on the supply air temperature and RH requirement. The preliminary work merged the main objective and penalties of requirement violations into a weighted reward function with constant penalty coefficients.…”

Section: Air Free Cooling Controlmentioning

confidence: 99%

See 1 more Smart Citation

Deep Reinforcement Learning for Tropical Air Free-Cooled Data Center Control

Wang

Liu

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Air free-cooled data centers (DCs) have not existed in the tropical zone due to the unique challenges of yearround high ambient temperature and relative humidity (RH). The increasing availability of servers that can tolerate higher temperatures and RH due to the regulatory bodies' prompts to raise DC temperature setpoints sheds light upon the feasibility of air free-cooled DCs in tropics. However, due to the complex psychrometric dynamics, operating the air free-cooled DC in tropics generally requires adaptive control of supply air condition to maintain the computing performance and reliability of the servers. This paper studies the problem of controlling the supply air temperature and RH in a free-cooled tropical DC below certain thresholds. To achieve the goal, we formulate the control problem as Markov decision processes and apply deep reinforcement learning (DRL) to learn the control policy that minimizes the cooling energy while satisfying the requirements on the supply air temperature and RH. We also develop a constrained DRL solution for performance improvements. Extensive evaluation based on real data traces collected from an air free-cooled testbed and comparisons among the unconstrained and constrained DRL approaches as well as two other baseline approaches show the superior performance of our proposed solutions. CCS Concepts: • Hardware → Enterprise level and data centers power issues; • Computing methodologies → Reinforcement learning.

show abstract

“…Based on the trained system dynamics, the operation of the HVAC system was managed by model predictive control to minimize both the energy cost and the indoor temperature constraints violation. The DRL approach was also applied to data centers with servers, which aim to minimize the energy used for moving air and on-demand cooling in the data centers through the control of the temperature and relative humidity of air supplied to the server [32].…”

Section: Introductionmentioning

confidence: 99%

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Lee

Choi

2020

Sensors

View full text Add to dashboard Cite

This paper presents a hierarchical deep reinforcement learning (DRL) method for the scheduling of energy consumptions of smart home appliances and distributed energy resources (DERs) including an energy storage system (ESS) and an electric vehicle (EV). Compared to Q-learning algorithms based on a discrete action space, the novelty of the proposed approach is that the energy consumptions of home appliances and DERs are scheduled in a continuous action space using an actor–critic-based DRL method. To this end, a two-level DRL framework is proposed where home appliances are scheduled at the first level according to the consumer’s preferred appliance scheduling and comfort level, while the charging and discharging schedules of ESS and EV are calculated at the second level using the optimal solution from the first level along with the consumer environmental characteristics. A simulation study is performed in a single home with an air conditioner, a washing machine, a rooftop solar photovoltaic system, an ESS, and an EV under a time-of-use pricing. Numerical examples under different weather conditions, weekday/weekend, and driving patterns of the EV confirm the effectiveness of the proposed approach in terms of total cost of electricity, state of energy of the ESS and EV, and consumer preference.

show abstract

Control of Air Free-Cooled Data Centers in Tropics via Deep Reinforcement Learning

Cited by 23 publications

References 8 publications

Control for smart systems: Challenges and trends in smart cities

Control for smart systems: Challenges and trends in smart cities

Deep Reinforcement Learning for Tropical Air Free-Cooled Data Center Control

Energy Management of Smart Home with Home Appliances, Energy Storage System and Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach

Contact Info

Product

Resources

About