Deep reinforcement learning with planning guardrails for building energy demand response

Jang, Doseok; Spangher, Lucas; Nadarajah, Selvaprabu; Spanos, Costas J.

doi:10.1016/j.egyai.2022.100204

Cited by 6 publications

(6 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of the total of six research papers [45,46,50,51,57,66], the geographical location of the analyzed buildings was not specified; however, in three research papers [52,53,63], the geographical location is specified only in terms of climate zones, indicating a broader coverage of geographical areas.…”

Section: Literature Reviewmentioning

confidence: 99%

“…This analysis used a meticulously chosen sample of the most recently published scientific research papers that were published over the past three years. Figure 3 Of the total of six research papers [45,46,50,51,57,66], the geographical location of the analyzed buildings was not specified; however, in three research papers [52,53,63], the geographical location is specified only in terms of climate zones, indicating a broader coverage of geographical areas.…”

Section: Literature Reviewmentioning

confidence: 99%

“…These findings underscore the tangible benefits of demand response in the commercial sector, paving the way for more efficient energy management practices. Furthermore, advanced techniques, such as deep reinforcement learning and planning guardrails, are giving promising results, outperforming traditional time-of-use pricing strategies and delivering substantial cost savings for building energy demand response [45]. These advancements highlight the potential of intelligent algorithms for optimizing energy consumption and curtailing energy costs.…”

Section: Key Findingsmentioning

confidence: 99%

See 2 more Smart Citations

Demand Response in Buildings: A Comprehensive Overview of Current Trends, Approaches, and Strategies

Jurjevic,

Zakula

2023

Buildings

View full text Add to dashboard Cite

Power grids in the 21st century face unprecedented challenges, including the urgent need to combat pollution, mitigate climate change, manage dwindling fossil fuel reserves, integrate renewable energy sources, and meet greater energy demand due to higher living standards. These challenges create heightened uncertainty, driven by the intermittent nature of renewables and surges in energy consumption, necessitating adaptable demand response (DR) strategies. This study addresses this urgent situation based on a statistical analysis of recent scientific research papers. It evaluates the current trends and DR practices in buildings, recognizing their pivotal role in achieving energy supply–demand equilibrium. The study analysis provides insight into building types, sample sizes, DR modeling approaches, and management strategies. The paper reveals specific research gaps, particularly the need for more detailed investigations encompassing building types and leveraging larger datasets. It underscores the potential benefits of adopting a multifaceted approach by combining multiple DR management strategies to optimize demand-side management. The findings presented in this paper can provide information to and guide future studies, policymaking, and decision-making processes to assess the practical potential of demand response in buildings and ultimately contribute to more resilient and sustainable energy systems.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Section: Key Findingsmentioning

confidence: 99%

See 1 more Smart Citation

Demand Response in Buildings: A Comprehensive Overview of Current Trends, Approaches, and Strategies

Jurjevic,

Zakula

2023

Buildings

View full text Add to dashboard Cite

show abstract

“…In this work, we lay the groundwork to accelerate our understanding of disruptions in tokamak plasmas through ML. In recent years, ML fields as diverse as natural language processing (NLP), computer vision (CV), energy systems, and robotics have seen impressive modeling advancements [19,20,21,22,23,24,25,26,27]; less visible have been the comprehensive benchmarking suites in each field that have laid the groundwork for these breakthroughs. These benchmarking suites standardize evaluation approaches for core tasks in each domain (e.g.…”

Section: Introductionmentioning

confidence: 99%

DisruptionBench: A robust benchmarking framework for machine learning-driven disruption prediction

Lucas,

Bonotto,

Arnold

et al. 2024

Preprint

View full text Add to dashboard Cite

Disruptions remain a major challenge to commercialization of fusion leveraging the tokamak configuration. Machine Learning (ML) holds promise for prediction in its flexibility and knowledge transfer between devices. However, the current body of ML-driven disruption prediction research lacks systematic benchmarking standards. This work introduces a novel benchmarking platform -- DisruptionBench -- encompassing nine modeling tasks tailored onto real-time tokamak operations: the cross of three test tokamaks (Alcator C-Mod, DIII-D, and EAST) with three test regimes (zero-shot, few-shot, and many-shot.) We conduct a comprehensive evaluation of four ML models re-implementing published ones as well as introducing two novel approaches – a GPT-2-like transformer and the Continuous Convolutional Neural Network (CCNN). Across all evaluation metrics, the CCNN demonstrates the most consistent and superior performance (best AUC=0.974), using the Alcator C-Mod tokamak as a test dataset. However, the GPT-2 architecture (best AUC=0.84) and the Hybrid Deep Learner (best AUC=0.92) re-implementation succeed in specific cases. Exploring the role of ''memory" in the transformer model, we observe that it often learns to recognize shot initiation, suggesting the potential utility of models with broader temporal context. Finally, as modeling demands for nuclear fusion datasets enter an era of unprecedented size and complexity, we characterize a newly compiled dataset spanning the entire operational history of Alcator C-Mod (1991-2016). These benchmarking, modeling, and data resources promise to accelerate the development of advanced ML-driven disruption prediction capabilities.

show abstract

“…Deep reinforcement learning (DRL) technologies have proven their effectiveness in complex system planning and control compared to other optimal control methods for many applications [1][2][3][4][5]. Real-time optimization solutions can be included in manufacturing processes [6], complex physical problems [7][8][9] where alternative approaches are computationally inefficient, and, as in our case, the optimal planning of experimental studies [10].…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning Environment Approach Based on Nanocatalyst XAS Diagnostics Graphic Formalization

Polyanichenko,

Protsenko,

Egil

et al. 2023

Materials

View full text Add to dashboard Cite

The most in-demand instrumental methods for new functional nanomaterial diagnostics employ synchrotron radiation, which is used to determine a material’s electronic and local atomic structure. The high time and resource costs of researching at international synchrotron radiation centers and the problems involved in developing an optimal strategy and in planning the control of the experiments are acute. One possible approach to solving these problems involves the use of deep reinforcement learning agents. However, this approach requires the creation of a special environment that provides a reliable level of response to the agent’s actions. As the physical experimental environment of nanocatalyst diagnostics is potentially a complex multiscale system, there are no unified comprehensive representations that formalize the structure and states as a single digital model. This study proposes an approach based on the decomposition of the experimental system into the original physically plausible nodes, with subsequent merging and optimization as a metagraphic representation with which to model the complex multiscale physicochemical environments. The advantage of this approach is the possibility to directly use the numerical model to predict the system states and to optimize the experimental conditions and parameters. Additionally, the obtained model can form the basic planning principles and allow for the optimization of the search for the optimal strategy with which to control the experiment when it is used as a training environment to provide different abstraction levels of system state reactions.

show abstract

Deep reinforcement learning with planning guardrails for building energy demand response

Cited by 6 publications

References 32 publications

Demand Response in Buildings: A Comprehensive Overview of Current Trends, Approaches, and Strategies

Demand Response in Buildings: A Comprehensive Overview of Current Trends, Approaches, and Strategies

DisruptionBench: A robust benchmarking framework for machine learning-driven disruption prediction

Deep Reinforcement Learning Environment Approach Based on Nanocatalyst XAS Diagnostics Graphic Formalization

Contact Info

Product

Resources

About