A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work?

Jiang, Daniel R.; Pham, Thuy Van; Powell, Warren B.; Salas, Daniel; Scott, Warren R.

doi:10.1109/adprl.2014.7010626

Cited by 42 publications

(39 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HAPI itself can be classified in this framework as a critic-only algorithm as it solely uses an approximation of the value function but not a separate approximation of the policy. Other modifications of HAPI could include, for example, the usage of different basis functions, such as higher order polynomials (see, e.g., Löhndorf and Minner 2010) and Gaussian radial basis functions (see, e.g., Jiang et al 2014). Likewise, researchers could investigate adapting this approach to a direct policy search (see, e.g., Scott and Powell 2012;Nascimento and Powell 2013) or to an approximate value iteration (AVI, see, e.g., Jiang and Powell 2015).…”

Section: Discussionmentioning

confidence: 99%

“…Surprisingly, in numerical experiments with real-world prices, they show that a very inefficient battery, which more or less only burns energy, has the highest value, because it exploits negative prices. Jiang et al (2014) directly compare the performance of different ADP approaches for the optimal control of an energy storage device. Jiang and Powell (2015) focus on one of these approaches and exploit the monotonicity of the value function.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Sell or store? An ADP approach to marketing renewable energy

Gönsch

Hassler

2016

OR Spectrum

View full text Add to dashboard Cite

In deregulated markets, electricity is usually traded in advance, and the advance commitments have a time lag of several periods. For example, in the German intraday market, the seller commits to providing electricity 45 min before the 15-min interval in which delivery has to be made. We consider the problem of a producer that generates energy from stochastic, renewable sources, such as solar or wind and uses a storage device with conversion losses. We model the problem as a Markov Decision Process and consider lagged commitments for the first time in the literature. The problem is solved using an innovative approximate dynamic programming approach. Its key elements are the analytical derivation of the optimal action based on the value function approximation and a new combination of approximate policy iteration with classical backward induction. The new approach is quite general with regard to the stochastic processes describing the energy production and price evolution. We demonstrate the application of our approach by considering a wind farm/storage combination. A numerical study using real-world data shows the applicability and performance of the new approach and investigates how the storage device's parameters influence profit.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Sell or store? An ADP approach to marketing renewable energy

Gönsch

Hassler

2016

OR Spectrum

View full text Add to dashboard Cite

show abstract

“…There has been recently a lot of interest in the development of efficient methods for stochastic optimal control problems such as stochastic gradient methods [23], the alternating directions method of multipliers (ADMM) [24] and various decomposition methods which can lead to parallelizable methods [25,26] (the most popular being the stochastic dual approximate dynamic programming [27], the progressive hedging approach [28] and dynamic programming [29]). There have been proposed parallelizable interior point algorithms for two-stage stochastic optimal control problems such as [30,31,32,33] and an ad hoc interior point solver for multi-stage problems [34].…”

Section: State Of the Artmentioning

confidence: 99%

Uncertainty-aware demand management of water distribution networks in deregulated energy markets

Sopasakis¹,

Sampathirao²,

Bemporad³

et al. 2018

Environmental Modelling & Software

View full text Add to dashboard Cite

We present an open-source solution for the operational control of drinking water distribution networks which accounts for the inherent uncertainty in water demand and electricity prices in the day-ahead market of a volatile deregulated economy. As increasingly more energy markets adopt this trading scheme, the operation of drinking water networks requires uncertainty-aware control approaches that mitigate the effect of volatility and result in an economic and safe operation of the network that meets the consumers' need for uninterrupted water supply. We propose the use of scenario-based stochastic model predictive control: an advanced control methodology which comes at a considerable computation cost which is overcome by harnessing the parallelization capabilities of graphics processing units (GPUs) and using a massively parallelizable algorithm based on the accelerated proximal gradient method.

show abstract

“…This is a parameterized cost function approximation where the reserve parameters are tuned in an online fashion (that is, the real world), although they could be tuned in a simulator (offline). • Value function approximations-Often referred to as dynamic programming (or approximate dynamic programming), value functions are particularly useful in the control of storage problems (see [34]- [36]). Value functions are widely approximated in the controls community using neural networks where they are often referred to as "critic nets."…”

Section: Illustrating the Four Classes In Energy Applicationsmentioning

confidence: 99%

Tutorial on Stochastic Optimization in Energy—Part I: Modeling and Policies

Powell¹,

Meisel

2016

IEEE Trans. Power Syst.

Self Cite

113

View full text Add to dashboard Cite

There is a wide range of problems in energy systems that require making decisions in the presence of different forms of uncertainty. The fields that address sequential, stochastic decision problems lack a standard canonical modeling framework, with fragmented, competing solution strategies. Recognizing that we will never agree on a single notational system, this two-part tutorial proposes a simple, straightforward canonical model (that is most familiar to people with a control theory background), and introduces four fundamental classes of policies which integrate the competing strategies that have been proposed under names such as control theory, dynamic programming, stochastic programming and robust optimization. Part II of the tutorial illustrates the modeling framework using a simple energy storage problem, where we show that, depending on the problem characteristics, each of the four classes of policies may be best. where he has taught since 1981. His research specializes in computational stochastic optimization, with applications in energy, transportation, health, and finance. He has authored/coauthored over 200 publications and two books. He founded and directs CASTLE Labs (http://www.castlelab.princeton.edu), specializing in fundamental contributions to computational stochastic optimization with a wide range of applications.

show abstract

A comparison of approximate dynamic programming techniques on benchmark energy storage problems: Does anything work?

Cited by 42 publications

References 24 publications

Sell or store? An ADP approach to marketing renewable energy

Sell or store? An ADP approach to marketing renewable energy

Uncertainty-aware demand management of water distribution networks in deregulated energy markets

Tutorial on Stochastic Optimization in Energy—Part I: Modeling and Policies

Contact Info

Product

Resources

About