Risk-averse dynamic programming for Markov decision processes

Tight Approximations of Dynamic Risk Measures

“…. , X T q into risk assessments at t. Following a large body of literature [Riedel, 2004, Artzner et al, 2007, Detlefsen and Scandolo, 2005, Roorda et al, 2005, Cheridito et al, 2006, Föllmer and Penner, 2006, Ruszczynski and Shapiro, 2006a, Ruszczyński, 2010, Cheridito and Kupper, 2011, we furthermore restrict the risk measurements at time t to only depend on the cumulative costs in the future, i.e., we take µ rt,T s : X T Ñ X t , and the risk of X rt,T s is µ rt,T s pX t`¨¨¨`XT q. While such measures have been criticized for ignoring the timing when future cashflows are received, they are consistent with the assumptions in many academic papers focusing on portfolio management under risk Chabakauri, 2010, Cuoco et al, 2008], as well as with current risk management practice [Jorion, 2006], and provide a natural, simpler first step in our analysis.…”

“…A central result in the literature [Riedel, 2004, Artzner et al, 2007, Detlefsen and Scandolo, 2005, Roorda et al, 2005, Cheridito et al, 2006, Roorda and Schumacher, 2007, Penner, 2007, Föllmer and Penner, 2006, Ruszczyński, 2010 is the following theorem, stating that any consistent measure has a compositional representation in terms of one-period risk mappings. Theorem 2.2.…”

“…Apart from being axiomatically justified, this compositional form has the advantage of allowing a recursive estimation of the risk, and an application of the Bellman optimality principle in optimization problems involving dynamic risk measures [Nilim and El Ghaoui, 2005, Iyengar, 2005, Ruszczynski and Shapiro, 2006a, Ruszczyński, 2010. This has lead to its adoption in actuarial science Wirch, 2004, Brazauskas et al, 2008], as well as in several recent papers that re-examine operational problems under coherent measures of risk [Ahmed et al, 2007, Choi et al, 2011.…”

“…. q˘, for a set of suitable mappings tµ τ u τ Ptt,...,T u (see, e.g., Epstein and Schneider [2003], Riedel [2004], Cheridito et al [2006], Artzner et al [2007], Roorda et al [2005], Föllmer and Penner [2006], Ruszczyński [2010]). Apart from yielding consistent preferences, this compositional form also allows a recursive estimation of the risk, and an application of the Bellman principle in optimization problems involving dynamic risk measures [Nilim and El Ghaoui, 2005, Iyengar, 2005, Ruszczynski and Shapiro, 2006a.…”

Optimal Information Blending with Measurements in the L2 Sphere

“…Robust solutions to linear programs with uncertainty have been extensively studied [5], and the theory of robust optimization has also been developed for Markov decision processes [30,25,36]. See also [38] for recent work connecting the robust solution and the uncertainty set to a risk measure chosen by the decision-maker. Particularly relevant to the present paper is [11], which derived an expression of the form (5) applied specifically to MDPs.…”