Privacy-Cost Management in Smart Meters Using Deep Reinforcement Learning

Shateri, Mohammadhadi; Messina, Francisco; Piantanida, Pablo; Labeau, Fabrice

doi:10.1109/isgt-europe47291.2020.9248831

Cited by 13 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reinforcement Learning [69][70][71][72][73][74][75][76][77][78] MDP [70,79] Bargain Theory [41][42][43] Price Theory [35][36][37] Contract Theory [45][46][47] Auction Theory [29-31]…”

Section: Simulation Models 1 Incentive Modelsmentioning

confidence: 99%

“…on the smart grid with three micro-grids each equipped with wind turbines show that this scheme increases the micro-grid utility compared to the existing schemes. Shateri et al [76] proposed a deep reinforcement learning algorithm named deep double Q-learning to manage the privacy cost in smart meters during energy trading in the smart grid. Wang et al [77] proposed an energy trading model based on the repeated game in which each micro-grid chooses its approach individually and randomly for trading and maximize its revenue.…”

Section: ) Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Energy Trading in the Smart Grid: Taxonomy, Research Challenges and Solutions

et al. 2021

View full text Add to dashboard Cite

The smart grid is generally studied as an efficient and powerful electric grid. With the assistance of information and communication technology (ICT), the electric grid can increase the performance of the power grid system with smart energy management. On the other hand, with the usage of renewable energy resources (RERs), smart energy storage, and new transmission technologies in the power grid system, various new features such as real-time monitoring, fast restoration, battery displays, automated outage management, etc. have been assimilated into the smart grid. These new features generate more complexity in energy transmission and constitute important challenges like low energy consumption, high energy cost, social welfare, etc. while designing energy trading mechanisms in the smart grid. In the Internet-of-Things (IoT) era, several scenarios such as micro-grids, energy harvesting networks, and vehicle-to-grid (V2G) networks are present where energy trading plays an important role. However, in these scenarios, there are energy transmission and distribution, security and privacy, energy consumption, system reliability, the criticality of data delivery, and a few more challenges caused by distrust, non-transparent, and uncertain energy markets. Motivated from these challenges, we present a four-layered architecture of energy trading used in the smart grid. We propose a comprehensive background regarding the main concepts of energy trading and the implication of enabling technologies that manage the energy imbalances in the smart grid. Then, we present a problem taxonomy based on incentive, mathematical, and simulation modeldriven approaches, which are widely used to control and maintain the energy trading mechanisms. Based on the findings from the literature, we also present a solution taxonomy with enabling technologies such as Energy Internet, Software-defined networking (SDN), and blockchain. In the end, a summary of future research directions based on the energy trading mechanisms is explored to provide deep insights to the readers.

show abstract

“…Reinforcement Learning [69][70][71][72][73][74][75][76][77][78] MDP [70,79] Bargain Theory [41][42][43] Price Theory [35][36][37] Contract Theory [45][46][47] Auction Theory [29-31]…”

Section: Simulation Models 1 Incentive Modelsmentioning

confidence: 99%

Section: ) Reinforcement Learningmentioning

confidence: 99%

A Survey on Energy Trading in the Smart Grid: Taxonomy, Research Challenges and Solutions

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A substantial amount of studies on SMs privacy were conducted, which can be classified in two main families: (i) SMs data manipulation techniques [5]- [11]; and (ii) user's demand load shaping approaches [12]- [22]. On the one hand, in the first family of methods, the consumers' load data are manipulated by a noisy transformation before sharing it to the UP.…”

Section: B Related Workmentioning

confidence: 99%

“…In some recent studies, physical resources are used to minimize the average relative difference between the grid load and a constant (or piecewise constant to accommodate a timevarying energy price) target load [20]- [22], i.e., to flatten the electricity consumption reported by the SMs. In [22], following the formulation in [21], the SMs privacy problem is cast as a Markov decision process (MDP) and a modelfree deep reinforcement learning (RL) algorithm known as the deep double Q-learning (DDQL) method is used to tackle this MDP problem. Even though this framework has been shown to be useful in limiting the leakage of sensitive information, the effectiveness of the flatness-based privacy measure remains unclear.…”

Section: B Related Workmentioning

confidence: 99%

Privacy-Cost Management in Smart Meters with Mutual Information-Based Reinforcement Learning

Shateri¹,

Messina²,

Piantanida³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Smart meters (SMs) share fine-grained electricity consumption of households with utility providers almost in realtime. This can violate the users' privacy since sensitive information is leaked through the SMs data. In this study, a novel privacyaware method which exploits the availability of a rechargeable battery (RB) is proposed. It is based on a Markov decision process (MDP) formulation in which the reward received by the agent is designed to control the trade-off between privacy and electricity cost. To obtain a robust and general privacy measure, we adopt the mutual information (MI) between the user's demand load and the masked load seen by the grid. Unlike previous studies, we model the whole temporal correlation in the data to estimate the MI in its general form. The training of the agent is done using a model-free deep reinforcement learning algorithm known as the deep double Q-learning (DDQL) method. In order to estimate the MI-based privacy signal, a neural network termed the Hnetwork is included in the scheme. The performance of the DDQL-MI algorithm is assessed empirically using actual SMs data and compared with simpler privacy measures. The results show significant improvements over the state-of-the-art privacyaware SMs methods.

show abstract

“…Many of the most recent studies of this family, incorporated information theoretic measures such as Mutual Information (MI) or Directed Information (DI) to model the amount of information leaked about the sensitive attributes and used Machine Learning (ML) algorithms for their implementation. On the other hand, the DLS approaches use physical resources such as rechargeable batteries, electric vehicles, and even renewable energy resources, to shape the users' power consumption to mask the sensitive patterns [14]- [25]. Recently, Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) methods have been used to tackle this problem, showing good performance against strong ML based attackers.…”

Section: Introductionmentioning

confidence: 99%

Real-Time Privacy-Preserving Data Release for Smart Meters

Shateri

Messina

Piantanida

et al. 2020

IEEE Trans. Smart Grid

Self Cite

View full text Add to dashboard Cite

Fine-grained Smart Meters (SMs) data recording and communication has enabled several features of Smart Grids (SGs) such as power quality monitoring, load forecasting, fault detection, and so on. In addition, it has benefited the users by giving them more control over their electricity consumption. However, it is wellknown that it also discloses sensitive information about the users, i.e., an attacker can infer users' private information by analyzing the SMs data. In this study, we propose a privacy-preserving approach based on non-uniform down-sampling of SMs data. We formulate this as the problem of learning a sparse representation of SMs data with minimum information leakage and maximum utility. The architecture is composed of a releaser, which is a recurrent neural network (RNN), that is trained to generate the sparse representation by masking the SMs data, and an utility and adversary networks (also RNNs), which help the releaser to minimize the leakage of information about the private attribute, while keeping the reconstruction error of the SMs data minimum (i.e., maximum utility). The performance of the proposed technique is assessed based on actual SMs data and compared with uniform down-sampling, random (non-uniform) down-sampling, as well as the state-of-the-art in privacy-preserving methods using a data manipulation approach. It is shown that our method performs better in terms of the privacy-utility trade-off while releasing much less data, thus also being more efficient.

show abstract

Privacy-Cost Management in Smart Meters Using Deep Reinforcement Learning

Cited by 13 publications

References 18 publications

A Survey on Energy Trading in the Smart Grid: Taxonomy, Research Challenges and Solutions

A Survey on Energy Trading in the Smart Grid: Taxonomy, Research Challenges and Solutions

Privacy-Cost Management in Smart Meters with Mutual Information-Based Reinforcement Learning

Real-Time Privacy-Preserving Data Release for Smart Meters

Contact Info

Product

Resources

About