Data Persistence in Large-Scale Sensor Networks with Decentralized Fountain Codes

Abstract

It may not be feasible for sensor networks monitoring nature and inaccessible geographical regions to include powered sinks with Internet connections. We consider the scenario where sinks are not present in large-scale sensor networks, and unreliable sensors have to collectively resort to storing sensed data over time on themselves. At a time of convenience, such cached data from a small subset of live sensors may be collected by a centralized (possibly mobile) collector. In this paper, we propose a decentralized algorithm using fountain codes to guarantee the persistence and reliability of cached data on unreliable sensors. With fountain codes, the collector is able to recover all data as long as a sufficient number of sensors are alive.We use random walks to disseminate data from a sensor to a random subset of sensors in the network. Our algorithms take advantage of the low decoding complexity of fountain codes, as well as the scalability of the dissemination process via random walks. We have proposed two algorithms based on random walks. Our theoretical analysis and simulation-based studies have shown that, the first algorithm maintains the same level of fault tolerance as the original centralized fountain code, while introducing lower overhead than naive random-walk based implementation in the dissemination process. Our second algorithm has lower level of fault tolerance than the original centralized fountain code, but consumes much lower dissemination cost.

On the reliability of large-scale distributed systems – A topological view

“…The dynamic and self-organizing nature of large-scale distributed systems brings more challenges to this issue. Over the past few years, numerous methods and techniques have been proposed for monitoring, modeling, detecting failures [8,9,10], and various schemes have been developed to improve the availability and the system resiliency [11,12,13,14,15] to provide reliable services.…”

“…This school of approaches mainly focuses on network anomaly and dynamic failure and is not beneficial to normal cases. Some other recently proposed algorithms also address the reliability problems of overlay networks [11,12,13,14], but they treat all the nodes equally without distinguishing the "critical" nodes from the ordinary ones. Indeed, critical nodes do exist in all sorts of distributed systems, such as critical nodes in service [16], in data or resources [11], and in topology.…”

Network Coding Applications

“…A decentralized implementation of Fountain codes for distributed data storage that uses geographic routing was proposed by Dimakis et al [15,16]. Another approach that uses random walks with traps to disseminate the source packets throughout the network was proposed by Lin et al [52]. A technique called growth codes to increase data persistence in wireless sensor networks by increasing the amount of information that can be recovered at the sink was introduced by Kamara et al [43].…”

Raptor packets: A packet-centric approach to distributed raptor code design

“…Collecting any subset of Raptor packets slightly larger than the size of the distributed network data content is sufficient for successful data recovery with high probability. As opposed to recently proposed node-centric distributed rateless coding schemes, where network nodes are responsible for collecting information packets and performing rateless encoding [5]- [7], the proposed approach is packetcentric as the encoding task is controlled by the Raptor packets themselves. Creating Raptor packets is a two-step procedure that corresponds to precoding and LT-coding step in standard Raptor codes.…”

“…It is applicable in many networking scenarios such as data gathering [5], data persistence [4] or distributed network storage [3][6] [7]. The efficiency of the proposed scheme is shown to approach closely the performance of centralized Raptor codes.…”