On Small World Graphs in Non-uniformly Distributed Key Spaces

Abstract

In this paper

Opportunities from open source search

“…Structured P2P networks limit the routing latency by O(logN ) number of hops, while the routing information maintained by each peer is also limited to O(logN ), where N is the total number of peers in a network. A recent analysis shows that such logarithmic-style networks exhibit properties of small-world networks capable of supporting non-uniformly distributed resource keys (which is the case in information retrieval), while preserving good load-balancing properties [8]. Open-source large-scale search can particularly benefit from P2P self-organisation and fault-tolerance since P2P systems require minimal in place infrastructure and maintenance, which significantly reduces costs compared to centralised solutions.…”

“…While we initially have a simple two-stage pipeline of crawler and basic information extraction, we envisage more complex pipelines in the future, for instance different linguistic or named entity modules and page or site authority ranking modules will be incorporated. The Open Archives Initiative Protocol for Metadata Harvesting 8 (OAI-PMH) is a flexible standard for delivering batches of marked-up documents, and several good open source libraries implement it. It can be used in a straight linear pipeline, or in a more complex processing network with offshoots or side-tracks.…”

On Routing in Distributed Hash Tables

“…In the following proofs, we deal with the skewed id space I by stretching it to the uniform space I as in [8]. The uniform space I is equivalent to the hop space v)) and id hop (u) = round(n·id (u)).…”

“…figure 1). The resulting network has small-world characteristics [8,12], which guarantees an expected routing cost of O(log n) between any two peers in the network. …”

The Essence of P2P: A Reference Architecture for Overlay Networks

“…For logarithmic-style overlay approaches, [8] shows that these approaches form graphs according to Kleinberg's smallworld principles [11]. It is proven that such graphs belong to the special class of "routing efficient" small-world networks where decentralized greedy search algorithms provide the best performance.…”