LH*—a scalable, distributed data structure

Abstract

We present a scalable distributed data structure called LH*. LH* generalizes Linear Hashing (LH) to distributed RAM and disk files. An LH* file can be created from records with primary keys, or objects with OIDs, provided by any number of distributed and autonomous clients. It does not require a central directory, and grows gracefully, through splits of one bucket at a time, to virtually any number of servers. The number of messages per random insertion is one in general, and three in the worst case, regardless of the file size. The number of messages per key search is two in general, and four in the worst case. The file supports parallel operations, e.g., hash joins and scans. Performing a parallel operation on a file of M buckets costs at most 2M ϩ 1 messages, and between 1 and O(log 2 M) rounds of messages.We first describe the basic LH* scheme where a coordinator site manages bucket splits, and splits a bucket every time a collision occurs. We show that the average load factor of an LH* file is 65-70% regardless of file size, and bucket capacity. We then enhance the scheme with load control, performed at no additional message cost. The average load factor then increases to 80 -95%. These values are about that of LH, but the load factor for LH* varies more.We next define LH* schemes without a coordinator. We show that insert and search costs are the same as for the basic scheme. The splitting cost decreases on the average, but becomes more variable, as cascading splits are needed to prevent file overload. Next, we briefly describe two variants of splitting policy, using parallel splits and presplitting that should enhance performance for high-performance applications.All together, we show that LH* files can efficiently scale to files that are orders of magnitude larger in size than single-site files. LH* files that reside in main memory may also be much faster than single-site disk files. Finally, LH* files can be more efficient than any distributed file with a centralized directory, or a static parallel or distributed hash file.

Scalable Store of Java Objects Using Range Partitioning

“…On the other side, scalability, high speed of operation and fault tolerance are important advantages of SDDS [8,7,10,11]. Storing an object-oriented database in SDDS should increase performance of OODBMS and make OODB applications scalable.…”

“…There are numerous architectures of SDDS file such as RP* [7], LH* [8], DDH [9], etc. The paper concerns a development of object oriented version of RP*N architecture to store Java objects in serialized form (OORP*N).…”

Online Balancing of Range-Partitioned Data with Applications to Peer-to-Peer Systems

“…Litwin et al [20,19] consider the design of scalable distributed data structures (SDDS) which share many features with the design philosophy of P2P systems, including the absence of centralization, and the ability to gracefully add or remove servers. Most work on SDDS has focused on hash partitioning, either at the tuple level [20] or at a block level [19].…”

“…Most work on SDDS has focused on hash partitioning, either at the tuple level [20] or at a block level [19]. Our work is complementary in that it can be utilized to enable true range partitioning for SDDS.…”

A Very Efficient Order Preserving Scalable Distributed Data Structure

“…There are several SDDS proposals in the literature: defining structures based on hashing techniques [3,9,12,16,17], on order preserving techniques [1,2,4,7,8,10], or for multi-dimensional data management techniques [11,14], and many others.…”