Making Peer-to-Peer Keyword Searching Feasible Using Multi-level Partitioning

Abstract

Abstract-This paper discusses large scale keyword searching on top of peer-to-peer (P2P) networks. The state-of-the-art keyword searching techniques for unstructured and structured P2P systems are query flooding and inverted list intersection respectively. However, it has been demonstrated that P2P-based large scale full-text searching is not feasible by using either of the two techniques. We propose in this paper a new index partitioning and building scheme, multi-level partitioning (MLP), and discuss its implementation on top of P2P networks. MLP can dramatically reduce bisection bandwidth and end-user latency compared to the partition-by-keyword scheme. And compare to partition-bydocument, it need only broadcast a query to moderate number of peers to generate precise results.

P2P Web Search Technology

“…有的文献将基于文档的索引切分和基于关键词的索引切分分别称作水平切分和垂直切分 [3] 或本地索引和 全局索引 [44] .Zhong 等人对这两类切分方式进行了评价和对比 [4] .另外,还有一类混合索引切分策略,其本质是将 上述两种切分方式结合起来,多层次索引切分 [45] 就是这样一种切分方式. 在已有的数据集选择方法中,比较有影响的主要有:Callan 等人提出的数据集排序方法 CORI [46] ,Fuhr 提出 的基于决策论的数据集选择框架 [47] ,Gravano 等人提出的 Gloss 方法 [48] ,Si 等人提出的基于统计语言模型的方 法 [49] .这些方法都没有考虑数据集之间的重叠关系,而只是独立地考虑每个数据集对查询结果的贡献,而考虑数 据集之间的重叠关系可以进一步提高数据集选择方法的有效性.Byers 等人提出了一种基于置换的技术,可以有 效地评估数据集之间的相似度 [50] ;Florescu 等人则使用概率测度对数据集之间的重叠和覆盖关系进行建模 [51] ;…”

Query Processing to Efficient Search in Ubiquitous Computing

“…Multi-level partitioning (MLP) transmits the query only to some limited number of peers to decrease the traffic. In order to select appropriate peers, the technique divides the node space into k logical groups [1]. [2,3] uses Bloom Filter and previous search results to compress the intermediate URI list.…”

OverCite: A Cooperative Digital Research Library

“…Furthermore, the Eternity Service [3] uses peer-to-peer technology to resist censorship of electronic documents. There have also been a number of systems for searching large data sets [5,9,16,19,29,38,43,44] and crawling the Web [7,10,28,39] using peer-to-peer systems. We share with these systems a desire to distribute work across many nodes to avoid centralized points of weakness and increase robustness and scalability.…”

“…If query scalability becomes an issue, we plan to explore techniques from recent DHT search proposals [5,16,19,29,38,43] or unstructured peer-to-peer search optimizations [9,44].…”