Convergent Numerical Scheme for Singular Stochastic Control with State Constraints in a Portfolio Selection Problem

Budhiraja, Amarjit; Ross, Kevin

doi:10.1137/050640515

Cited by 33 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next we will introduce "stretched-out" time scale. This is similar to the approach previously used by Kushner [KM91] and Budhiraja and Ross [BR07] for singular control problems. Using the new time scale, we can overcome the possible non-tightness of the family of processes {Y h (·), Z h (·)} h>0 .…”

Section: (B) Define the Non-intervention Regionmentioning

confidence: 58%

Harvesting of interacting stochastic populations

et al. 2019

View full text Add to dashboard Cite

We analyze the optimal harvesting problem for an ecosystem of species that experience environmental stochasticity. Our work generalizes the current literature significantly by taking into account non-linear interactions between species, state-dependent prices, and species injections. The key generalization is making it possible to not only harvest, but also 'seed' individuals into the ecosystem. This is motivated by how fisheries and certain endangered species are controlled. The harvesting problem becomes finding the optimal harvesting-seeding strategy that maximizes the expected total income from the harvest minus the lost income from the species injections. Our analysis shows that new phenomena emerge due to the possibility of species injections.It is well-known that multidimensional harvesting problems are very hard to tackle. We are able to make progress, by characterizing the value function as a viscosity solution of the associated Hamilton-Jacobi-Bellman (HJB) equations. Moreover, we provide a verification theorem, which tells us that if a function has certain properties, then it will be the value function. This allows us to show heuristically, as was shown in Lungu and Øksendal (Bernoulli '01), that it is almost surely never optimal to harvest or seed from more than one population at a time.It is usually impossible to find closed-form solutions for the optimal harvesting-seeding strategy. In order to by-pass this obstacle we approximate the continuous-time systems by Markov chains. We show that the optimal harvesting-seeding strategies of the Markov chain approximations converge to the correct optimal harvesting strategy. This is used to provide numerical approximations to the optimal harvesting-seeding strategies and is a first step towards a full understanding of the intricacies of how one should harvest and seed interacting species. In particular, we look at three examples: one species modeled by a Verhulst-Pearl diffusion, two competing species and a two-species predator-prey system.

show abstract

Section: (B) Define the Non-intervention Regionmentioning

confidence: 58%

Harvesting of interacting stochastic populations

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In addition, for problems such as singular stochastic control and impulsive control, the HJB equations are in fact a system of partial differential inequalities. The existence, uniqueness of viscosity solutions and regularity theory for this class of PDEs are not well understood [38].…”

Section: Numerical and Algorithmic Methodsmentioning

confidence: 99%

- Residual Gas and Stochastic Control

Shen¹,

Zhang²,

Jiao³

et al. 2015

Transient Control of Gasoline Engines

View full text Add to dashboard Cite

Controlling dynamical systems in uncertain environments is fundamental and essential in several fields, ranging from robotics, healthcare to economics and finance. In these applications, the required tasks can be modeled as continuous-time, continuous-space stochastic optimal control problems. Moreover, risk management is an important requirement of such problems to guarantee safety during the execution of control policies. However, even in the simplest version, finding closed-form or exact algorithmic solutions for stochastic optimal control problems is comuputationally challenging.The main contribution of this thesis is the development of theoretical foundations, and provably-correct and efficient sampling-based algorithms to solve stochastic optimal control problems in the presence of complex risk constraints.In the first part of the thesis, we consider the mentioned problems without risk constraints. We propose a novel algorithm called the incremental Markov Decision Process (iMDP) to compute incrementally any-time control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as am incrementally refined model of the original problem. We show that the iMDP algorithm guarantees asymptotic optimality while maintaining low computational and space complexity.In the second part of the thesis, we consider risk constraints that are expressed as either bounded trajectory performance or bounded probabilities of failure. For the former, we present the first extended iMDP algorithm to approximate arbitrarily well an optimal feedback policy of the constrained problem. For the latter, we present a martingale approach that diffuses a risk constraint into a martingale to construct time-consistent control policies. The martingale stands for the level of risk tolerance that is contingent on available information over time. By augmenting the system dynamics with the martingale, the original risk-constrained problem is transformed into a stochastic target problem. We present the second extended iMDP algorithm to approximate arbitrarily well an optimal feedback policy of the original problem by sampling in the augmented state space and computing proper boundary values for the reformulated problem. In both cases, sequences of policies returned from the extended algorithms are both probabilistically sound and asymptotically optimal.

show abstract

“…For a convergence analysis of a numerical scheme for multidimensional singular control problems in the diffusion setting using Markov chain approximation methods, let us mention Kushner and Martins [20] and Budhiraja and Ross [9]; see also the book of Kushner and Dupuis [19] for an exhaustive survey. Regarding convergence of numerical schemes using the viscosity solution approach, let us mention for instance Souganidis [25] and Barles and Souganidis [8], where they propose a numerical scheme for non-singular control problems in the context of the diffusion setting; roughly speaking, they prove that the solutions of the numerical scheme converge to a viscosity solution of the associated HJB equation and then, using a uniqueness argument, they obtain the convergence result.…”

Section: Introductionmentioning

confidence: 99%

A Multidimensional Problem of Optimal Dividends with Irreversible Switching: A Convergent Numerical Scheme

Azcue

Muler

2019

Appl Math Optim

View full text Add to dashboard Cite

In this paper we study the problem of optimal dividend payment strategy which maximizes the expected discounted sum of dividends to a multidimensional set up of n associated insurance companies where the surplus process follows an n-dimensional compound Poisson process. The general manager of the companies has the possibility at any time to exercise an irreversible switch into another regime; we also take into account an expected discounted value at ruin. This multidimensional dividend problem is a mixed singular control/optimal problem. We prove that the optimal value function is a viscosity solution of the associated HJB equation and that it can be characterized as the smallest viscosity supersolution. The main contribution of the paper is to provide a numerical method to approximate (locally uniformly) the optimal value function by an increasing sequence of sub-optimal value functions of admissible strategies defined in an n-dimensional grid. As a numerical example, we present the optimal time of merger for two insurance companies.

show abstract

Convergent Numerical Scheme for Singular Stochastic Control with State Constraints in a Portfolio Selection Problem

Cited by 33 publications

References 27 publications

Harvesting of interacting stochastic populations

Harvesting of interacting stochastic populations

- Residual Gas and Stochastic Control

A Multidimensional Problem of Optimal Dividends with Irreversible Switching: A Convergent Numerical Scheme

Contact Info

Product

Resources

About