Redundancy control through traffic deduplication

Abstract

Statistics show that 79% of the Internet traffic is video and mostly "redundant". Video-On-Demand in particular follows a 90/10 access pattern, where 90% of the users access the same 10% of all video content. As a result, redundant data are repeatedly transmitted over the Internet. In this paper, we propose a novel traffic deduplication technique to achieve more efficient network communication between video sources (video servers or proxy servers in a CDN) and clients. The proposed SMART (Small packet Merge-Able RouTers) overlay network employs an opportunistic traffic deduplication approach and allows each SMART router to dynamically merge independent streams of the same video content, forming a video streaming tree (VST). The merged streams are tunneled through the overlay together with TCP sessions information before eventually being de-multiplexed and delivered to the clients fully compatible with the TCP protocol. We present theoretical analysis findings on the merging strategy between the video source and clients, the efficiency of the SMART router to save traffic during a merge process, and the overall performance of implementing a SMART overlay topology between a video source and clients. Finally, we prototyped SMART in the PlanetLab environment. We illustrate that performance evaluation results are consistent with our theoretical analysis and significant bandwidth saving is achieved.