Prospective Experiment for Reinforcement Learning on Demand Response in a Social Game Framework

Spangher, Lucas; Gokul, Akash; Khattar, Manan; Palakapilly, Joseph; Tawade, Akaash; Bouyamourn, Adam; Devonport, Alex; Spanos, Costas J.

doi:10.1145/3396851.3402365

Cited by 9 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of many architectures we explored, we present Soft Actor Critic V2 here (the "controller" in Figure 1.) We skip a description of how the architectures are structured an function; please see [9] for a tutorial.…”

Section: Reinforcement Learning Techniquesmentioning

confidence: 99%

“…Each model is made and trained on a simulated dataset of two months worth of energy and points data to mirror conditions of the experiment. The training dataset we used and a full explanation of the planning models we tested are detailed in [9]. Of various AutoML strategies, we present here the top performing GPyOpt LSTM search [3]; we compare this to an Oracle model (i.e., calls the same function as the simulation) an OLS and a Baseline model that returns a single value.…”

Section: Planning Models: Improving Rl Agent's Data Efficiencymentioning

confidence: 99%

“…We define three deterministic responses to points, and populate our office with an equal mixture "people" who exhibit each response type. We will describe the linear response here and then refer the reader to [9] for definition of sinusoidal and threshold exponential response, as they are simple variations.…”

Section: Office Workers Responsementioning

confidence: 99%

See 2 more Smart Citations

Augmenting Reinforcement Learning with a Planning Model for Optimizing Energy Demand Response

Spangher

Gokul

Khattar

et al. 2020

Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings &Amp; Cities

Self Cite

View full text Add to dashboard Cite

While reinforcement learning (RL) on humans has shown incredible promise, it often suffers from a scarcity of data and few steps. In instances like these, a planning model of human behavior may greatly help. We present an experimental setup for the development and testing of an Soft Actor Critic (SAC) V2 RL architecture for several different neural architectures for planning models: an autoML optimized LSTM, an OLS, and a baseline model. We present the effects of including a planning model in agent learning within a simulation of the office, currently reporting a limited success with the LSTM. CCS CONCEPTS• Hardware → Smart grid; Enterprise level and data centers power issues; Impact on the environment; • Applied computing → Multicriterion optimization and decision-making.

show abstract

Section: Reinforcement Learning Techniquesmentioning

confidence: 99%

Section: Planning Models: Improving Rl Agent's Data Efficiencymentioning

confidence: 99%

See 1 more Smart Citation

Augmenting Reinforcement Learning with a Planning Model for Optimizing Energy Demand Response

Spangher

Gokul

Khattar

et al. 2020

Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings &Amp; Cities

Self Cite

View full text Add to dashboard Cite

show abstract

“…As the grid decarbonizes, volatile resources like wind and solar will replace on-demand resources like fossil fuels, and there arises a mismatch between generation and demand. Grids that do not prepare for this question will face daunting consequences, from curtailment of resources [22] to voltage instability and physical damage, despite having adequate generative capability. Indeed, these problems will only grow larger as the world moves away from fossil fuels.…”

Section: Introductionmentioning

confidence: 99%

“…The SinBerBEST collaboration has developed a Social Game [10] that facilitates workers to engage in a competition around energy [9], [3]. Through this framework, a first-of-its-kind experiment has been proposed to implement behavioral demand response within an office building [22]. Prior work has proposed to describe an hourly price-setting controller that learns how to optimize its prices [21].…”

Section: Introductionmentioning

confidence: 99%

Using Meta Reinforcement Learning to Bridge the Gap between Simulation and Experiment in Energy Demand Response

Jang

Spangher

Khattar

et al. 2021

Proceedings of the Twelfth ACM International Conference on Future Energy Systems

Self Cite

View full text Add to dashboard Cite

Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we apply a meta-learning architecture to warm start the experiment with simulated tasks, to increase sample efficiency. We present results that demonstrate a similar a step up in complexity still corresponds with better learning. CCS CONCEPTS• Hardware → Energy distribution; • Computer systems organization → Neural networks; • Theory of computation → Reinforcement learning.

show abstract

Machine Learning for Smart and Energy-Efficient Buildings

Das,

Lin,

Agwan

et al. 2024

Environ. Data Science

Self Cite

View full text Add to dashboard Cite

Energy consumption in buildings, both residential and commercial, accounts for approximately 40% of all energy usage in the United States, and similar numbers are being reported from countries around the world. This significant amount of energy is used to maintain a comfortable, secure, and productive environment for the occupants. So, it is crucial that energy consumption in buildings must be optimized, all the while maintaining satisfactory levels of occupant comfort, health, and safety. Machine learning (ML) has been proven to be an invaluable tool in deriving important insights from data and optimizing various systems. In this work, we review some of the most promising ways in which ML has been leveraged to make buildings smart and energy-efficient. For the convenience of readers, we provide a brief introduction to the relevant ML paradigms and the components and functioning of each smart building system we cover. Finally, we discuss the challenges faced while implementing machine learning algorithms in smart buildings and provide future avenues for research in this field.

show abstract

Prospective Experiment for Reinforcement Learning on Demand Response in a Social Game Framework

Cited by 9 publications

References 7 publications

Augmenting Reinforcement Learning with a Planning Model for Optimizing Energy Demand Response

Augmenting Reinforcement Learning with a Planning Model for Optimizing Energy Demand Response

Using Meta Reinforcement Learning to Bridge the Gap between Simulation and Experiment in Energy Demand Response

Machine Learning for Smart and Energy-Efficient Buildings

Contact Info

Product

Resources

About