Optimal Plans for Dynamic Programming Problems

Abstract

It is proved that there exist stationary optimal plans for discounted dynamic programming problems, and that there exist semi-Markov ϵ-optimal plans for positive dynamic programming problems. The actions are required to be taken in a variable action set F(s), and the reward function r(s, a) is a Borel measurable function of (s, a) and an u.s.c. function of a. Our results are related to recent work by Furukawa, Maitra, and Schal. The key tool is a generalization of a selection theorem of Dubins and Savage.

N—Person Stochastic Games: Extensions of the Finite State Space Case and Correlation

“…We assume that B is a Borel subset of S × Y whose projection on the horizontal axis is S. From Theorem 3 in [22], we obtain the following auxiliary result.…”

“…and f * λ (s) belongs to the convex hull of N w * (s) for every s ∈ S. Using standard results on discounted dynamic programming [5], [22], we infer that…”

Three principles of competitive nonlinear pricing

“…We shall refer to any direct mechanism (i(¢); f(¢)) satisfying the incentive compatibility constraints (8) as well as the individual rationality constraints (9) as a competitively viable direct mechanism (or as a CVD mechanism). Note that because each contract set K i (for i = 0; 1; : : : ; m) contains a "no contracting" choice, endogenous participation by …rms and the agent is automatically built into the notion of competitive viability.…”

“…8 Under a competitively viable direct contracting mechanism the agent will participate in the mechanism and choose the …rm-contract pair intended by the mechanism precisely because, by design, the mechanism provides no incentives 6 In general, the agent's best response mapping is upper semicontinuous in catalog pro…les (rather than continuous), and each …rm's payo¤ function is upper semicontinuous in contracts (rather than continuous). The connection between nonexistence of Nash equilibria and lack of continuity in competitive contracting games was …rst illustrated via an example by Myerson (1982).…”

“…7 Reny's notion of payo¤ security provides us with a precise way of identifying allowable discontinuities. 8 Thus, here unlike Martimort and Stole (1997) we assume that the agent can contract with one and only one …rm.…”