We establish the existence of an optimal control for a general class of singular control problems with state constraints. The proof uses weak convergence arguments and a time rescaling technique. The existence of optimal controls for Brownian control problems [14], associated with a broad family of stochastic networks, follows as a consequence. . This reprint differs from the original in pagination and typographic detail. 1 2 A. BUDHIRAJA AND K. ROSS process as a constrained diffusion with reflection at the free boundary. Excepting specific models (cf. [30, 31]), this approach encounters substantial difficulties, even for linear dynamics (cf. [32]); a key difficulty is that little is known about the regularity of the free boundary in higher dimensions. Alternative approaches for establishing the existence of optimal controls based on compactness arguments are developed in [12,17,25]. The first of these papers considers linear dynamics, while the last two consider models with nonlinear coefficients. In all three papers, the state space is all of R d , that is, there are no state constraints. It is important to note that in the current paper, although the drift and diffusion coefficients are constant, the state constraint requirement introduces a (nonstandard) nonlinearity into the dynamics. To the best of our knowledge, the current paper is the first to address the existence of an optimal control for a general multidimensional singular control problem with state constraints. While our method does not provide any characterization of the optimal control, it is quite general and should be applicable to other families of singular control problems (with or without state constraints). State constraints are a natural feature in many practical applications of singular control. A primary motivation for the problems considered in this paper arises from applications in controlled queueing systems. Under "heavy traffic conditions", formal diffusion approximations of a broad family of queuing networks with scheduling control lead to the so-called Brownian control problems (BCP's) (cf. [14]). The BCP can in turn be transformed, by applying techniques introduced by Harrison and Van Mieghem [16] to a singular control problem with state constraints. We refer the reader to [1] for a concise description of the connections between Brownian control problems and the class of singular control problems studied in [1] and the current paper. In the Appendix, we indicate how the results of the current paper lead to the existence of optimal controls for BCP's. State constraints arise in numerous other applications: see Davis and Norman [10] and Duffie, Fleming, Soner and Zariphopoulou [11] (and references therein) for control problems with state constraints in mathematical finance.In Section 2, we define the singular control problem of interest. The main result of this paper (Theorem 2.3) establishes the existence of an optimal control. An important application of such a result lies in establishing connections between singular control proble...

Abstract. We consider a singular stochastic control problem with state constraints that arises in problems of optimal consumption and investment under transaction costs. Numerical approximations for the value function using the Markov chain approximation method of Kushner and Dupuis are studied. The main result of the paper shows that the value function of the Markov decision problem (MDP) corresponding to the approximating controlled Markov chain converges to that of the original stochastic control problem as various parameters in the approximation approach suitable limits. All our convergence arguments are probabilistic; the main assumption that we make is that the value function be finite and continuous. In particular, uniqueness of the solutions of the associated HJB equations is neither needed nor available (in the generality under which the problem is considered). Specific features of the problem that make the convergence analysis nontrivial include unboundedness of the state and control space and the cost function; degeneracies in the dynamics; mixed boundary (Dirichlet-Neumann) conditions; and presence of both singular and absolutely continuous controls in the dynamics. Finally, schemes for computing the value function and optimal control policies for the MDP are presented and illustrated with a numerical study.

A singular stochastic control problem with state constraints in twodimensions is studied. We show that the value function is C 1 and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the noaction region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133-2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not C 1 . This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is C 2 in the interior of the no-action region. We then use a verification argument applied to a suitable C 2 approximation of the value function to establish optimality of the conjectured control.