We study the convergence of Markov Decision Processes made of a large number of objects to optimization problems on ordinary differential equations (ODE). We show that the optimal reward of such a Markov Decision Process, satisfying a Bellman equation, converges to the solution of a continuous Hamilton-Jacobi-Bellman (HJB) equation based on the mean field approximation of the Markov Decision Process. We give bounds on the difference of the rewards, and a constructive algorithm for deriving an approximating solution to the Markov Decision Process from a solution of the HJB equations. We illustrate the method on three examples pertaining respectively to investment strategies, population dynamics control and scheduling in queues are developed. They are used to illustrate and justify the construction of the controlled ODE and to show the gain obtained by solving a continuous HJB equation rather than a large discrete Bellman equation.

Performance evaluation of the 802.11 MAC protocol is classically based on the decoupling assumption, which hypothesizes that the backoff processes at different nodes are independent. This decoupling assumption results from mean field convergence and is generally true in transient regime in the asymptotic sense (when the number of wireless nodes tends to infinity), but, contrary to widespread belief, may not necessarily hold in stationary regime. The issue is often related with the existence and uniqueness of a solution to a fixed point equation; however, it was also recently shown that this condition is not sufficient; in contrast, a sufficient condition is a global stability property of the associated ordinary differential equation. In this paper, we give a simple condition that establishes the asymptotic validity of the decoupling assumption for the homogeneous case (all nodes have the same parameters). We also discuss the heterogeneous and the differentiated service cases and formulate a new ordinary differential equation. We show that the uniqueness of a solution to the associated fixed point equation is not sufficient; we exhibit one case where the fixed point equation has a unique solution but the decoupling assumption is not valid in the asymptotic sense in stationary regime.

Abstract-Energy-detection (ED) receivers can take advantage of the ranging and multipath resistance capabilities of impulseradio ultra-wideband (IR-UWB) physical layers at a much lower complexity than coherent receivers. However, ED receivers are extremely vulnerable to multi-user interference (MUI). Therefore, the design of IR-UWB ED architectures must take MUI into account. In this paper, we present the design and evaluation of two complementary algorithms for reliable and robust synchronization of IR-UWB ED receivers in the presence of MUI: 1) powerindependent detection and preamble code interference cancellation (PICNIC) and 2) detection of start-frame-delimiter through sequential ratio tests (DESSERT). PICNIC addresses packet detection and timing acquisition while DESSERT focuses on startframe-delimiter (SFD) detection. Both algorithms are evaluated with the IEEE 802.15.4a IR-UWB physical layer, standardized for low data-rate networks. The performance evaluation with extensive simulations show that our algorithms outperform nonrobust synchronization algorithms by up to two orders of magnitude in the presence of MUI.Index Terms-IEEE 802.15 standards, interference cancellation, multiple-access interference, ultra-wideband communication.