Markov decision processes with multidimensional action spaces

Pandelis, Dimitrios G.

doi:10.1016/j.ejor.2009.01.039

Cited by 5 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process modeling methodology is based on a Markov chain (Eckert et al, 2004; Flanagan et al, 2007; Pandelis, 2010). A Markov chain consists of a set of nodes linked by directed arcs.…”

Section: Process Modeling Methodologymentioning

confidence: 99%

Bayesian project diagnosis for the construction design process

Matthews

Philip

Dong³

et al. 2012

AIEDAM

View full text Add to dashboard Cite

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. Abstract This study demonstrates how subtle signals taken from the early stages within a construction process can be used to diagnose potential problems within that process. For this study, the construction process is modelled as a quasi-Markov chain. A set of six different scenarios representing various common problems (e.g. small budget, complex project) are created and simulated by suitably defining the transition probabilities between nodes in the Markov chain. A Monte Carlo approach is used to parametrise a Bayesian estimator. By observing the time taken to pass the Review Gateway (as measured by the number of hops between activity nodes), the system is able to determine with good accuracy the problem scenario that the construction process will likely suffer from.

show abstract

“…The process modeling methodology is based on a Markov chain (Eckert et al, 2004; Flanagan et al, 2007; Pandelis, 2010). A Markov chain consists of a set of nodes linked by directed arcs.…”

Section: Process Modeling Methodologymentioning

confidence: 99%

Bayesian project diagnosis for the construction design process

Matthews

Philip

Dong³

et al. 2012

AIEDAM

View full text Add to dashboard Cite

show abstract

“…The value of this type of approach is that they can lead to exact optimal policies in the long run under a stochastic context (Pandelis, 2010;Zhuang and Li, 2012;Fianu and Davis, 2018). As mentioned above, chats are distributed among the agents so as to maximize the average service rate per chat.…”

Section: The Service Initiation Problem In the Swcmentioning

confidence: 99%

On the scheduling of operations in a chat contact center

Legros¹,

Jouini

2019

European Journal of Operational Research

View full text Add to dashboard Cite

Unlike calls, the chat channel allows a contact center agent to simultaneously work with many customers. The benefit of this flexibility can be however challenged by a new abandonment feature during service. In this context, new flow routing questions arise. How many chats should an agent serve? Which agent should be selected? Or may a chat be served by more than one agent? We aim to answer these questions so as to find the best trade-off between time spent in service, queueing delay and abandonment.For this purpose, we determine conditions under which the traditional Least Busy First or Most Busy First policies are optimal for agent selection. Using a dynamic programming analysis, we prove that a state-dependent threshold reservation policy is optimal when a chat can be handed to other agents without loss of efficiency. This analysis reveals that reservation is useful as it allows to maintain of sufficient agent productivity and abandonment from the queue which in turn reduce the system's congestion. In addition, we show the closeness between the optimal policy and the easier-to-implement fixed capacity policy if this fixed capacity is optimized. When a chat cannot be handed to other agents, we develop a dispatching policy improvement algorithm based on the explicit computation of the relative value function for an initial policy with fixed probabilities. The dispatching policy significantly outperforms non state-dependent ones, as it partly compensates the agents' inability to share a chat service.

show abstract

Relevant states and memory in Markov chain bootstrapping and simulation

Cerqueti

Falbo

Pelizzari

2017

European Journal of Operational Research

View full text Add to dashboard Cite

Markov chain theory is proving to be a powerful approach to bootstrap highly nonlinear time series. In this work we provide a method to estimate the memory of a Markov chain (i.e. its order) and to identify its relevant states. In particular the choice of memory lags and the aggregation of irrelevant states are obtained by looking for regularities in the transition probabilities. Our approach is based on an optimization model. More specifically we consider two competing objectives that a researcher will in general pursue when dealing with bootstrapping:preserving the "structural" similarity between the original and the simulated series and assuring a controlled diversification of the latter. A discussion based on information theory is developed to define the desirable properties for such optimal criteria. Two numerical tests are developed to verify the effectiveness of the method proposed here.MSC classification: 60J10, 60J20, 60J22, 62B10, 62F40, 91G60.

show abstract

Markov decision processes with multidimensional action spaces

Cited by 5 publications

References 7 publications

Bayesian project diagnosis for the construction design process

Bayesian project diagnosis for the construction design process

On the scheduling of operations in a chat contact center

Relevant states and memory in Markov chain bootstrapping and simulation

Contact Info

Product

Resources

About