Backward Approximate Dynamic Programming with Hidden Semi-Markov Stochastic Models in Energy Storage Optimization

Abstract

We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, capturing the amount of time each process stays above or below some benchmark such as the forecast. This is an important property of stochastic processes involved in storage problems. We show that we achieve more robust solutions using this model than when more common stochastic models are considered. The new model introduces some additional complexity to the problem as its information states are partially hidden, forming a partially observable Markov decision process. We derive a near-optimal time-dependent policy using backward approximate dynamic programming, which overcomes the computational hurdles of classical (exact) backward dynamic programming, with higher quality solutions than the more familiar forward approximate dynamic programming methods.

Optimal Daily Trading of Battery Operations Using Arbitrage Spreads

“…The stochasticity of electricity prices, wind power output and load are important aspects of all these high frequency adaptive models. In Durante et al [52], the stochastic prices and wind power forecast errors have been modeled by a hidden semi-Markov model. This model can accurately capture the error distribution as well as the crossing times to represent when each process is above or below a certain benchmark, for example the forecast.…”

Green Hydrogen Plant: Optimal control strategies for integrated hydrogen storage and power generation with wind energy

“…Similarly, Löhndorf et al (2013) consider bidding on a short-term intraday market and a long-term interday market. Hassler (2017) present a model for short-term trading with a time lag between trade and delivery and Durante et al (2017) use forecasted power output with forecast errors. Qi et al (2015) focus on the optimal size and sites of energy storage systems, as well as the associated topology and capacity of the transmission network under a given policy instrument.…”

Risk Directed Importance Sampling in Stochastic Dual Dynamic Programming with Hidden Markov Models for Grid Level Energy Storage

“…3. With distributed storage devices and the exogenous wind process modeled with the univariate crossing state model from Durante et al (2017). In this case the version of SDDP from Algorithm 2 without sampling is utilized (i.e.…”

“…in Durante et al (2017), but is repeated here for the convenience of the reader. The following is presented in the context of using the model for wind power forecast errors, as is done in this paper.…”

“…Additionally, the distribution of areas above and below the forecast (the surpluses and deficits of energy produced versus expected output) are accurately replicated by the crossing state model. In Durante et al (2017) we see that modeling these characteristics is indeed important in storage problems as control policies based on value function approximations (VFAs) for a single storage device developed assuming a crossing state model for wind power are shown to be more robust than those trained with standard ARMA models. Here we look to apply this model to produce robust policies in a high dimensional setting with many storage devices spread across a power grid.…”