We consider time-average Markov Decision Processes (MDPs), which accumulate a reward and cost at each decision epoch. A policy meets the sample-path constraint if the time-average cost is below a specified value with probability one. The optimization problem is to maximize the expected average reward over all policies that meet the sample-path constraint. The sample-path constraint is compared with the more commonly studied constraint of requiring the average expected cost to be less than a specified value. Although the two criteria are equivalent for certain classes of MDPs, their feasible and optimal policies differ for many nontrivial problems. In general, there does not exist optimal or nearly optimal stationary policies when the expected average-cost constraint is employed. Assuming that a policy exists that meets the sample-path constraint, we establish that there exist nearly optimal stationary policies for communicating MDPs. A parametric linear programming algorithm is given to construct nearly optimal stationary policies. The discussion relies on well known results from the theory of stochastic processes and linear programming. The techniques lead to simple proofs of the existence of optimal and nearly optimal stationary policies for unichain and deterministic MDPs, respectively.

We consider finite-state finite-action Markov decision processes which accumulate both a reward and a cost at each decision epoch. We study the problem of finding a policy that maximizes the expected long-run average reward subject to the constraint that the long-run average cost be no greater than a given value with probability one. We establish that if there exists a policy that meets the constraint, then there exists an ε-optimal stationary policy. Furthermore, an algorithm is outlined to locate the ε-optimal stationary policy. The proof of the result hinges on a decomposition of the state space into maximal recurrent classes and a set of transient states.

Abstract. Partitioning and allocation of relations is an important component of the distributed database design. Several approaches (and algorithms) have been proposed for clustering data for pattern classification and for partitioning relations in distributed databases. Most of the approaches used for classification use square-error criterion. In contrast, most of the approaches proposed for partitioning of relations are either ad hoc solutions or solutions for special cases (e.g., binary vc.r tical partitioning).In this paper, we first highlight the differences between the approaches taken for pattern classification and for distributed databases. Then an objective function for vertical partitioning of relations is derived using the square-error criterion commonly used in data clustering. The objective function derived generalizes and subsumes earlier work on vertical partitioning. Furthermore, the approach proposed in this paper is shown to be useful for comparing previously developed algorithms for vertical partitioning. The objective function has also been extended to include additional information, such as transaction types, different local and remote accessing costs and replication. Finally, we discuss the implementation of a distributed database design testbed.

No abstract

No abstract