Decentralized and locality aware replication method for DHT-based P2P storage systems

Abstract

Skip Graph, a type of DHT, plays an important role in P2P cloud storage applications, where nodes publicly or privately store, share, and access data. Nowadays P2P storage systems are widely using replication to support data availability, reliability, and maintainability. With replication, the main consideration is determining peers to replicate the data. Traditional replication algorithms are partially randomized and employ rigid assumptions about nodes' distribution. This results in high access delay between nodes and their closest replicas, which degrades the system performance. We propose GLARAS, a dynamic and fully decentralized locality aware replication method for Skip Graph. In contrast to the traditional algorithms, which replicate based on strict assumptions about the distribution of nodes, GLARAS aims to approximate the underlying distribution by interacting with a very small subset of nodes and minimize the average access delay of replication accordingly. To ensure GLARAS performs at its best, we also propose a dynamic fully decentralized landmark-based locality aware name ID assignment namely LANS. This ensures that the nodes' distances in the overlay and the underlying network are consistent with each other. Our extensive experiments and analysis results demonstrate that compared to the best existing decentralized locality aware replication, GLARAS improves the average access delay of public and private replications by about 13% and 17%, respectively. Likewise, in comparison to the best existing decentralized locality aware name ID assignment, LANS improves the locality awareness of name IDs and the end-to-end latency of search queries in Skip Graph with the gains of about 19% and 8%, respectively. The average replication's access delay of a Skip Graph-based P2P storage system that employs GLARAS and LANS has an improvement gain of about 2.7 over the best state-of-the-art algorithms. Since Skip Graph is a DHT, any other DHT-based P2P storage service would benefit from our solution.

Detouring Skip Graph: Efficient Routing via Detour Routes on Skip Graph Topology

“…Thus, arithmetic operations and absolute values are well-defined on K. Moreover, we also assume that there is at most one node for each key. In practice, the uniqueness of keys is not always guaranteed because replication [26], [27] is often used to provide data availability and fault tolerance. A simple way to eliminate duplicate keys is to add some random bits on the right side of identical keys [28].…”

“…3 and for this mid definition, mid (9, 18) = 27 2 holds, and thus closeToRight(k target , 9, 18) ⇐⇒ 27 2 < k target . This means the target node is located to the right of the estimated center if 27 2 < k target , and is located to the left if k target ≤ 27 2 . Next, if…”

The Management Strategy of Metadata in Large-scale Network Storage System

“…The rapid development of modern computer network technology and the increasing transmission bandwidth provide the possibility to use the network to build a storage system with large capacity, high reliability and scalability. Thus, recently large-scale network storage systems based on structured overlay networks and distributed hash tables have become a new trend in the development of distributed storage technologies [5]. Because such systems often contain a large number of nodes and data, how to efficiently store and retrieve the required metadata in the system to locate the corresponding nodes and data is the cornerstone and key to the construction of the system.…”

Inserção de Infraestrutura de Chave Pública no Projeto OpenDHT

“…Dessa forma, elaé atrativa para uma diversidade de aplicações como em [Rodrigues et al 2016], em que a rede DHTé utilizada para propagação de regras de firewall. Por outro lado, em [Hassanzadeh-Nazarabadi et al 2018], a tecnologiaé usada para redundância de dados. [Eisele et al 2017] usa a rede DHT para prover um serviço de descoberta de aplicações e [Hassanzadeh-Nazarabadi et al 2019] utiliza para implementação de uma Blockchain em ambientes com recursos computacionais limitados.…”