In this paper, we study the M n /M n /c/K + M n queueing system where customers arrive according to a Poisson process with state-dependent rates. Moreover, the rates of the exponential service times and times to abandonment of the queued customers can also change whenever the system size changes. This implies that a customer may experience different service rates throughout the time she is being served. Similarly, a queued customer can change her patience time limits while waiting in the queue. Thus, we refer to the analyzed system as the "sensitive" Markovian queue. We conduct an exact analysis of this system and obtain its steady-state performance measures. The steady-state system size distribution yields itself via a birth-death process. The times spent in the queue by an arbitrary or an eventually served customer are represented as the times until absorption in two continuous-time Markov chains and follow Phase-type distributions with which the queueing time distributions and moments are obtained. Then, we demonstrate how the M n /M n /c/K + M n queue can be employed to approximately yet accurately estimate the performance measures of the M n /GI/c/K + GI type call center.

We consider the problem of sequential sampling from a finite number of independent statistical populations to maximize the expected infinite horizon average outcome per period, under a constraint that the expected average sampling cost does not exceed an upper bound. The outcome distributions are not known. We construct a class of consistent adaptive policies, under which the average outcome converges with probability 1 to the true value under complete information for all distributions with finite means. We also compare the rate of convergence for various policies in this class using simulation.

We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activation) from a population (bandit) incurs a known bandit dependent cost. Successive samples from each population are iid random variables with unknown distribution. The objective is to design a feasible policy for deciding from which population to sample from, so as to maximize the expected sum of outcomes of n total samples or equivalently to minimize the regret due to lack on information on sample distributions, For this problem we consider the class of feasible uniformly fast (f-UF) convergent policies, that satisfy the cost constraint sample-path wise. We first establish a necessary asymptotic lower bound for the rate of increase of the regret function of f-UF policies. Then we construct a class of f-UF policies and provide conditions under which they are asymptotically optimal within the class of f-UF policies, achieving this asymptotic lower bound. At the end we provide the explicit form of such policies for the case in which the unknown distributions are Normal with unknown means and known variances. . Asymptotic optimality, finite horizon regret bounds, and a solution to an open problem. optimal Bayesian sequential change detection and identification rules. -armed bandit with budget constraint and variable costs. In AAAI-13, pages 232-238, 2013.Eugene A Feinberg, Pavlo O Kasyanov, and Michael Z Zgurovsky. Convergence of value iterations for total-cost mdps and pomdps with general state and action sets.

We consider a two-product inventory system with independent Poisson demands, limited joint storage capacity and partial demand substitution. Replenishment is performed simultaneously for both products and the replenishment time may be fixed or exponentially distributed. For both cases we develop a Continuous Time Markov Chain model for the inventory levels and derive expressions for the expected profit per unit time. We establish analytic expressions for the profit function and show that it satisfies decreasing differences properties in the order quantities, which allows for a more efficient algorithm to determine the optimal ordering policy. Using computational experiments, we assess the effect of substitution and replenishment time uncertainty on the order quantities and the profit as a function of the storage capacity.

No abstract