A reinforcement learning approach for optimal heating curve adaption

Huang, Chenzi; Seidel, Stephan; Paschke, Fabian; Bräunig, Jan

doi:10.1109/etfa52439.2022.9921461

Cited by 2 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To obtain learning data for the reinforcement learning based control design, a simplified Modelica model of the building's upper floor and its heating network is developed according to [10]. The modelling tool SimulationX and the GreenCity Library [12] is used.…”

Section: Modellingmentioning

confidence: 99%

“…The following system variables can be defined as the states for the supply temperature control [10]: Current ambient temperature 𝑠𝑠 𝑇𝑇 𝑎𝑎 , current zone temperature 𝑠𝑠 𝑇𝑇 𝑧𝑧 , current hour of day 𝑠𝑠 𝐻𝐻𝐻𝐻𝐻𝐻 , current week day 𝑠𝑠 𝑊𝑊 , and predicted solar energy of an upcoming time horizon 𝑠𝑠 𝐹𝐹𝐹𝐹𝐻𝐻𝐹𝐹 , as well as predicted mean ambient temperature upcoming time horizon 𝑠𝑠 𝐹𝐹𝐹𝐹𝑇𝑇𝐹𝐹 . They give rise to a state vector 𝒔𝒔.…”

Section: Basic Q-learning Control Designmentioning

confidence: 99%

“…Eventually, a reward function needs to be defined. As the goal for the demonstrator building is to maximize comfort, the deviation of zone temperature 𝑇𝑇 𝑧𝑧𝐻𝐻𝑧𝑧𝑧𝑧 from its setpoint 𝑇𝑇 𝑧𝑧𝐻𝐻𝑧𝑧𝑧𝑧,𝑠𝑠 is computed [10],…”

Section: Basic Q-learning Control Designmentioning

confidence: 99%

“…In this contribution, a reinforcement learning based supply temperature control approach -Q-learning (QL) -is proposed. Based on the work in [10] the controller is designed such that occupation and weather prediction are taken into account. Further, in order to deal with thermal effects of different time constants, the control design is extended with a cascaded structure.…”

Section: Introductionmentioning

confidence: 99%

“…Its modelling is presented which is used to generate learning data for the control design. In Section 3 the reinforcement learning control from [10] is briefly described and simulation results are discussed. In Section 4 a cascaded control design is introduced and evaluated followed by simulation results.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Cascaded reinforcement learning based supply temperature control

Huang,

Seidel,

Bräunig

2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

In this work, a Q-learning based supply temperature control approach for a demonstrator building is proposed. The purpose is to improve the temperature behaviour inside the building and to tackle comfort problems such as overheating and undercooling which cannot be coped with by the standard heating curve. The Q-learning controller considers predicted future weather data as system states. This can be shown to be superior to Q-learning controllers without weather prediction. Furthermore, in order for the controller to capture different thermal effects of different time constants, a cascaded control structure is designed: An inner Q-learning based controller deals with thermal effects of smaller time constants. It is wrapped by an outer slower Q-learning controller which can tackle effects of larger time constants. Therewith, further improvement of comfort can be achieved.

show abstract

Section: Modellingmentioning

confidence: 99%

Section: Basic Q-learning Control Designmentioning

confidence: 99%

Section: Basic Q-learning Control Designmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cascaded reinforcement learning based supply temperature control

Huang,

Seidel,

Bräunig

2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

Model-Free Deep Reinforcement Learning for Adaptive Supply Temperature Control in Collective Space Heating Systems

Ghane,

Jacobs,

Huybrechts

et al. 2024

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

The conventional approach for controlling the supply temperature in collective space heating networks relies on a predefined heating curve determined by outdoor temperature and heat emitter type. This prioritizes thermal comfort but lacks energetic and financial optimization. This research proposes an adaptive supply temperature control in well-insulated dwellings, responsive to diverse environmental parameters. The approach considers variable electricity prices and accommodates different indoor temperature set points in dwellings. The study evaluates the effectiveness of two Deep Reinforcement Learning (DRL) algorithms, i.e. Proximal Policy Optimization (PPO) and Deep Q-Network (DQN), across various scenarios. Results reveal that DQN excels in collective space heating systems with underfloor heating in each dwelling, while PPO proves superior for radiator-based systems. Both outperform the traditional heating curve, achieving up to 13.77% (DQN) and 16.15% (PPO) cost reduction while guaranteeing thermal comfort. Additionally, the research highlights the capability of DRL-based methods to dynamically set the supply temperature based on a cloud of set points, showcasing adaptability to diverse environmental factors and addressing the growing significance of indoor heat gains in well-insulated dwellings. This innovative approach holds promise for more efficient and environmentally conscious heating strategies within collective space heating networks.

show abstract

A reinforcement learning approach for optimal heating curve adaption

Cited by 2 publications

References 0 publications

Cascaded reinforcement learning based supply temperature control

Cascaded reinforcement learning based supply temperature control

Model-Free Deep Reinforcement Learning for Adaptive Supply Temperature Control in Collective Space Heating Systems

Contact Info

Product

Resources

About