An MDP-Based Approach for Multipath Data Transmission over Wireless Networks

Abstract

Abstract-Maintaining performance and reliability in wireless networks is a challenging task due to the nature of wireless channels. Multipath data transmission has been used in wired scenarios to reduce latency, improve throughput, andwhen/where possible -balance the load. In this paper, we propose an approach for multipath data transmission over wireless networks. We demonstrate that the problem under study can be formulated as a Markov Decision Process (MDP) and we propose an algorithm called On-line Policy Iteration (OPI), to solve the formulated MDP in real time. We verified the proposed approach using simulations with ns-2 and data collected from real heterogeneous wired/wireless networks. The results indicate that we improve both delay and loss characteristics of end-to-end wireless communications outperforming the classical multi-path schemes including Round Robin and Join the Shortest Queue.

A Markovian Approach to Multipath Data Transfer in Overlay Networks

“…Preliminary results within the same framework of this work have been recently published in [11], [8], [10]. The research leading to these results has received partial funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement n. 224263-OneLab2 and from the project LATINO of the FARO programme financed by the Compagnia di San Paolo and by the Polo delle Scienze e delle Tecnologie of the University of Napoli Federico II.…”

“…In this paper, we propose an approach to distribute packets over multiple overlay paths, which is based on Markov Decision Process, and allows to achieve better performance than Weighted Round Robin (WRR) and Join the Shortest Queue (JSQ). Thanks to the existence of Markov property in many network characteristics, e.g., arrival processes, loss, and delay [6], [9], [25], [33], MDPs have been used in the networking field to address various decision problems (e.g., [3], [11]). However, this does not mean that applying an MDP framework is straightforward.…”

“…2, here the traffic distributor module is integrated into the end points. Each wireless path is comprised of three wireless nodes (two hosts and an Access Point), which implement Ricean radio propagation model, 11 802.11 MAC with the basic rate of 1 Mbps, 600 m of transmission range, and supporting Dynamic Source Routing Protocol [21]. To create disjoint wireless paths, different physical channels are used.…”

Evaluating “no-new-wires” home networks

“…Although flow based multi-path routing schemes can avoid packet reordering, they cannot fully utilize available bandwidth and cannot serve a flow that requires more bandwidth than the capacity of any single path. [19] and [20] have introduced a few multi-path scheduling algorithms. However, they focus on improving packet loss, delay and jitter rather than increasing system throughput by fully utilizing each individual path.…”

Online Packet Dispatching for Concurrent Transmissions in Heterogeneous Multi-RAT Networks

“…For example, an algorithm based on the Join-the-ShortestQueue (JSQ) policy is adopted in [14] to reduce the mean cost of packets over different multi-RATs. In [15], the authors propose an algorithm based on the Markov Decision Process (MDP) model, which is shown to outperform the classical Round Robin (RR) and JSQ based multi-path schemes.…”