On the convergence to stationarity of birth-death processes

Coolen‐Schrijner, Pauline; Doorn, Erik A. van

doi:10.1017/s0021900200018854

Cited by 8 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…with m j0 and m e0 given by (6.5) and (6.6), respectively. It is now easy to obtain the following Corollary to Theorem 7.1, which is the result obtained earlier in [2]. …”

Section: An Applicationsupporting

confidence: 68%

See 1 more Smart Citation

The Deviation Matrix of a Continuous-Time Markov Chain

Coolen‐Schrijner

Doorn

2002

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Abstract. The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P (.) and ergodic matrix Π is the matrixWe give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth-death process is investigated in detail. We also describe a new application of deviation matrices by showing that a measure for the convergence to stationarity of a stochastically increasing Markov chain can be expressed in terms of the elements of the deviation matrix of the chain.

show abstract

“…with m j0 and m e0 given by (6.5) and (6.6), respectively. It is now easy to obtain the following Corollary to Theorem 7.1, which is the result obtained earlier in [2]. …”

Section: An Applicationsupporting

confidence: 68%

“…The present authors [2] have recently evaluated (7.2) for birth-death processes in general. In the (more general) setting at hand m(X ) can be expressed in terms of the elements of the deviation matrix of X , as shown in the next theorem.…”

Section: An Applicationmentioning

confidence: 99%

The Deviation Matrix of a Continuous-Time Markov Chain

Coolen‐Schrijner

Doorn

2002

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For all schemes however B p (n) and B s (n) can take values 0 or 1, that is, condition 2 for the service processes holds. For all schemes, the service processes are either iid Bernoulli processes, or are controlled by the state of the queues, that in turn, when stable, can be described with ergodic discrete time birth-deaths processes, that converge monotonically to steady state [30]. (For example, under opportunistic spectrum sharing, the primary service process is an iid Bernoulli process with E[B p (n)] = q pd , while the secondary service process is controlled by the primary queue length…”

Section: ) If the St Performs Sequential Decisions π Then The Stablmentioning

confidence: 99%

Dynamic Cooperative Secondary Access in Hierarchical Spectrum Sharing Networks

Wang

Fodor²

2014

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

Abstract-We consider a hierarchical spectrum sharing network consisting of a primary and a cognitive secondary transmitter-receiver pair, with non-backlogged traffic. The secondary transmitter may utilize cooperative transmission techniques to relay primary traffic while superimposing its own information, or transmit opportunistically when the primary user is idle. The secondary user meets a dilemma in this scenario. Choosing cooperation it can transmit a packet immediately even if the primary queue is not empty, but it has to bear the additional cost of relaying, since the primary performance needs to be guaranteed. To solve this dilemma we propose dynamic cooperative secondary access control that takes the state of the spectrum sharing network into account. We formulate the problem as a Markov Decision Process (MDP) and prove the existence of a stationary policy that is average cost optimal. Then we consider the scenario when the traffic and link statistics are not known at the secondary user, and propose to find the optimal transmission strategy using reinforcement learning. With extensive numerical evaluation, we demonstrate that dynamic cooperation with state aware sequential decision is very efficient in spectrum sharing systems with stochastic traffic, and show that dynamic cooperation is necessary for the secondary system to be able to adapt to changing load conditions or to changing available energy resource. Our results show, that learning based access control, with or without known primary buffer state, has close to optimal performance. Index Terms-Hierarchical spectrum sharing, cooperative transmission, queuing systems, Markov decision process, reinforcement learning.

show abstract

“…However, there exists no general expression for α in terms of the birth and death rates. As an alternative measure, Stadje and Parthasarathy [16] and Coolen-Schrijner and Van Doorn [5] considered the quantities I = ∞ 0 [EX − EX(t)] dt and the normalized value…”

Section: Speed Of Convergence To Stationaritymentioning

confidence: 99%

Zipf and Lerch Limit of Birth and Death Processes

Klar¹,

Parthasarathy²,

Henze³

2009

Prob. Eng. Inf. Sci.

View full text Add to dashboard Cite

Birth and death processes are useful in a wide range of disciplines from computer networks and telecommunications to chemical kinetics and epidemiology. Data from many different areas such as linguistics, music, or warfare fit Zipf's law surprisingly well. The Lerch distribution generalizes Zipf's law and is applicable in survival and dispersal processes. In this article we construct a birth and death process that converges to the Lerch distribution in the limit as time becomes large, and we investigate the speed of convergence. This is achieved by employing continued fractions. Numerical illustrations are presented through tables and graphs.

show abstract

On the convergence to stationarity of birth-death processes

Cited by 8 publications

References 13 publications

The Deviation Matrix of a Continuous-Time Markov Chain

The Deviation Matrix of a Continuous-Time Markov Chain

Dynamic Cooperative Secondary Access in Hierarchical Spectrum Sharing Networks

Zipf and Lerch Limit of Birth and Death Processes

Contact Info

Product

Resources

About