Privacy-Preserving Energy Management of a Shared Energy Storage System for Smart Buildings: A Federated Deep Reinforcement Learning Approach

Abstract

This paper proposes a privacy-preserving energy management of a shared energy storage system (SESS) for multiple smart buildings using federated reinforcement learning (FRL). To preserve the privacy of energy scheduling of buildings connected to the SESS, we present a distributed deep reinforcement learning (DRL) framework using the FRL method, which consists of a global server (GS) and local building energy management systems (LBEMSs). In the framework, the LBEMS DRL agents share only a randomly selected part of their trained neural network for energy consumption models with the GS without consumer’s energy consumption data. Using the shared models, the GS executes two processes: (i) construction and broadcast of a global model of energy consumption to the LBEMS agents for retraining their local models and (ii) training of the SESS DRL agent’s energy charging and discharging from and to the utility and buildings. Simulation studies are conducted using one SESS and three smart buildings with solar photovoltaic systems. The results demonstrate that the proposed approach can schedule the charging and discharging of the SESS and an optimal energy consumption of heating, ventilation, and air conditioning systems in smart buildings under heterogeneous building environments while preserving the privacy of buildings’ energy consumption.

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

“…[21]). As another example, relevant state information for the electricity management of a building's HVAC (Heating, Ventilation and Air Conditioning) includes indoor temperatures and occupancy [22].…”

“…Nakabi & Toivanen [142] consider household loads that respond to a dynamic price signal, and assume that user comfort is incorporated to the load controllers as a price elasticity parameter. Lee et al [22] optimize a HVAC system and minimize of the deviations of the indoor environment outside an ideal range. Lee & Choi [130] and Yu et al [136] considers appliance agents for home energy management systems which are optimized against two criteria: reducing electricity bills while satisfying the consumer comfort level for heating and the consumer preferences for appliances.…”

“…A common battery modelling approach in the environment of the RL agent is to capture energy losses resulting from battery operations with factors for charging and discharging efficiency, and to impose limits to charging and discharging power (e.g. [24], [116], [133], [29], [142], [22]). Whereas most authors capture losses as a simple coefficient for charging and discharging efficiency, a few authors use more detailed models.…”

“…2) ENERGY COMMUNITY Communities of buildings offer further optimization opportunities with shared batteries. An aggregator can trade the capacity of the batteries and other flexible energy resources on utility markets [22], [145], [130]. Alternatively, a local market can be established to avoiding buying and selling from the grid [143], [158], [118], [14].…”

Comprehensive Study of the Microclimate Parameters in the Residential Building

“… analysis of thermal losses in buildings [3,4,9,14,18];  analysis of energy efficiency and flexibility of the air conditioning and heating [1,15];  creating and improving calculation methods, algorithms to achieve optimal and effective energy consumption during peak hours [7,8,13,17];  experimental research using modern materials and studying their impact on increasing energy efficiency of buildings [11,16];  developing heat indicators and energy efficiency class limits in buildings [2,5,10,12]. The research papers analyzed point to a lack of studies on the impact of the microclimate of residential buildings on their efficiency.…”