Parameterized penalties in the dual representation of Markov decision processes

2014

Quantitative Finance

Self Cite

Fast pricing of American-style options has been a difficult problem since it was first introduced to the financial markets in 1970s, especially when the underlying stocks' prices follow some jumpdiffusion processes. In this paper, we extend the 'true martingale algorithm' proposed by Belomestny et al. [Math. Finance, 2009, 19, 53-71] for the pure-diffusion models to the jump-diffusion models, to fast compute true tight upper bounds on the Bermudan option price in a non-nested simulation manner. By exploiting the martingale representation theorem on the optimal dual martingale driven by jump-diffusion processes, we are able to explore the unique structure of the optimal dual martingale and construct an approximation that preserves the martingale property. The resulting upper bound estimator avoids the nested Monte Carlo simulation suffered by the original primal-dual algorithm, therefore significantly improving the computational efficiency. Theoretical analysis is provided to guarantee the quality of the martingale approximation. Numerical experiments are conducted to verify the efficiency of our algorithm.

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fast estimation of true bounds on Bermudan option prices under jump-diffusion processes

Zhu

2014

Quantitative Finance

Self Cite

“…The naive approach is to replace the optimal value functions with approximate ones, and use nested simulation to estimate the conditional expectations; however, this approach often requires substantial computational effort and might cause the resulted approximation to lose the dual feasibility. Various methods have been proposed to improve the accuracy and efficiency of the approximation, including the non-nested simulation approach by Belomestny et al (2009) and Zhu et al (2015) in American-style option pricing, and the pathwise optimization techniques by Desai et al (2011) and Ye and Zhou (2012). The advantage of the pathwise optimization method is that it explores a subspace of feasible dual penalties by considering the best linear combination of the existing dual penalties.…”

Section: Introductionmentioning

confidence: 99%

Solving the Dual Problems of Dynamic Programs via Regression

Zhu

IEEE Trans. Automat. Contr.

2018

Self Cite

In recent years, information relaxation and duality in dynamic programs have been studied extensively, and the resulted primal-dual approach has become a powerful procedure in solving dynamic programs by providing lower-upper bounds on the optimal value function. Theoretically, with the so called value-based optimal dual penalty, the optimal value function could be recovered exactly via strong duality. However, in practice, obtaining tight dual bounds usually requires good approximations of the optimal dual penalty, which could be time-consuming due to the conditional expectations that need to be estimated via nested simulation. In this paper, we will develop a framework of regression approach to approximating the optimal dual penalty in a non-nested manner, by exploring the structure of the function space consisting of all feasible dual penalties. The resulted approximations maintain to be feasible dual penalties, and thus yield valid dual bounds on the optimal value function. We show that the proposed framework is computationally efficient, and the resulted dual penalties lead to numerically tractable dual problems. Finally, we apply the framework to a high-dimensional dynamic trading problem to demonstrate its effectiveness in solving the dual problems of complex dynamic programs.

“…In addition, the optimal penalty is not unique: for general problems we have the value function-based penalty developed by [7] and [8]; for problems with convex structure there is an alternative optimal penalty, that is, the gradient-based penalty, as pointed out by [11]. On the other hand, in order to derive tight dual bounds, various approximation schemes based on different optimal penalties have been proposed including [8], [11], [12], [13]. We notice that this dual approach has found increasing applications in different fields, such as [14], [11], [15], [16], [17].…”

Section: Introductionmentioning

confidence: 99%

Information Relaxation and Dual Formulation of Controlled Markov Diffusions

IEEE Trans. Automat. Contr.

2015

Self Cite

Information relaxation and duality in Markov decision processes have been studied recently by several researchers with the goal to derive dual bounds on the value function. In this paper we extend this dual formulation to controlled Markov diffusions: in a similar way we relax the constraint that the decision should be made based on the current information and impose a penalty to punish the access to the information in advance. We establish the weak duality, strong duality and complementary slackness results in a parallel way as those in Markov decision processes. We further explore the structure of the optimal penalties and expose the connection between the optimal penalties for Markov decision processes and controlled Markov diffusions. We demonstrate the use of this dual representation in a classic dynamic portfolio choice problem through a new class of penalties, which require little extra computation and produce small duality gap on the optimal value.