AnchorHash: A Scalable Consistent Hash

Mendelson, Gal; Vargaftik, Shay; Barabash, Katherine; Lorenz, Dean; Keslassy, Isaac; Orda, Ariel

doi:10.1109/tnet.2020.3039547

Cited by 18 publications

(17 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jump hashing [15] and AnchorHash [16] are algorithms that combine hash functions in multiple layers and reduce the number of hash calculations compared to HRW. Jump hashing [15] utilizes per-backend hash functions as well as HRW but reduces the number of per-lookup hash calculations to O(log(n)) on average by arranging the order of hash value evaluation.…”

Section: Related Workmentioning

confidence: 99%

“…However, the arrangement restricts the order of backend removals, so Jump hashing is not appropriate in environments like cloud infrastructure where the backend removal order is not controllable because of backend failures. With AnchorHash [16], the number of hash calculations can be reduced to 1 + ln(n max /n) on average and n max − n at the maximum. Here, n is the number of active backends, and n max is set to exceed the maximum number of backends, considering future needs.…”

Section: Related Workmentioning

confidence: 99%

“…Many methods have been proposed to improve load balancing of hashing with consistency [3], [7], [15], [16], [17], [20], [22], [24]. However, they have all struggled to overcome a trade-off among the following five properties, so most cloud infrastructure components still use original CH.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Structured Allocation-Based Consistent Hashing With Improved Balancing for Cloud Infrastructure

Nakatani

2021

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Consistent hashing has played an indispensable role in cloud infrastructure, although its load balancing performance is not necessarily perfect. Consistent hashing has long remained the most widely used method despite many methods being proposed to improve load balancing because these methods trade off load balancing against consistency, memory usage, lookup performance, and/or fault-tolerance. This paper presents Structured Allocation-based Consistent Hashing (SACH), a cloud-optimized consistent hashing algorithm that overcomes the trade-offs by taking advantage of the characteristics of cloud environments: scaling management and auto-healing. Since scaling can be distinguished from failures, SACH applies two different algorithms to update hashing functions: a fast-update algorithm for unmanaged backend failures to satisfy fault-tolerance with quick response and a slow-update algorithm for managed scaling. Hashing functions are initialized or slow-updated considering the characteristics of the fast-update algorithm to satisfy load balancing and the other properties as far as the number of failed backends is kept small by auto-healing. The experimental results show that SACH outperforms existing algorithms in each aspect. SACH will improve the load balancing of cloud infrastructure components, where the trade-offs have prevented the renewal of hashing functions.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Structured Allocation-Based Consistent Hashing With Improved Balancing for Cloud Infrastructure

Nakatani

2021

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…Although SACH makes a serious of optimizations, the memory footprint and the update performance is still not ideal. The Jump consistent hash [6] and AnchorHash [8] have a relatively ideal performance of the five properties. But the scalibility of these two CHes have limitations repectively.…”

Section: Introductionmentioning

confidence: 99%

DxHash: A Scalable Consistent Hash Based on the Pseudo-Random Sequence

Dong¹,

Wang²

2021

Preprint

View full text Add to dashboard Cite

Consistent hasing has played a fundamental role as a data router and a load balancer in various fields, such as distributed database, cloud infrastructure, and peer-to-peer network. However, the existing consistent hashing schemes can't meet the requirements simultaneously, including full consistency, scalability, small memory footprint, low update time and low query complexity. Thus, We propose DxHash, a scalable consistent hashing algorithm based on the pseudo-random sequence. For the scenario of distributed storage, there are two optimizations based on DXHash are proposed. First, the Weighted DxHash can adjust the workloads on arbitrary nodes. Second, the Asymmetric Replica Strategy (ARS) is combining the replica strategy in distributed storage with the scaleup process to improve the availability of the system and reduce the remapping rate. The evaluation indicates that compared with the state-of-art works, DxHash achieves significant improvements on the 5 requirements. Even with 50% failure ratio, DxHash still can complete 16.5 million queries per second. What's more, the two optimizations both achieve their own results.

show abstract

“…With help of limit and offset our technique seeks the index file and loads the required meta-data to the memory. To access index file randomly may be an expensive operation in case of the large index files therefore to limit the size of index files another special hash function: Scalable-Spittable Hash Function (SSHF) [3] [4] [5] is used that will dynamically distribute the meta-data of the massive number of small files to the various index file in place of the single index file. The remaining section of the paper is as follows; Section 2 presents the literature review on the existing techniques to deal with small file problem.…”

Section: Introductionmentioning

confidence: 99%

A Novel Technique for Handling Small File Problem of HDFS: Hash Based Archive File (HBAF)

Sharma¹,

Barwar²

2021

Recent Trends in Intensive Computing

View full text Add to dashboard Cite

Now a day’s, Data is exponentially increasing with the advancement in the data science. Each and every digital footprint is generating enormous amount of data, which is further used for processing various tasks to generate important information for different end user applications. To handle such enormous amount of data, there are number of technologies available, Hadoop/HDFS is one of the big data handling technology. HDFS can easily handle the large files but when there is the case to deal with massive number of small files, the performance of the HDFS degrades. In this paper we have proposed a novel technique Hash Based Archive File (HBAF) that can solve the small file problem of the HDFS. The proposed technique is capable to read the final index files partly, that will reduce the memory load on the Name Node and offer the file appending capability after creation of the archiv.

show abstract

AnchorHash: A Scalable Consistent Hash

Cited by 18 publications

References 10 publications

Structured Allocation-Based Consistent Hashing With Improved Balancing for Cloud Infrastructure

Structured Allocation-Based Consistent Hashing With Improved Balancing for Cloud Infrastructure

DxHash: A Scalable Consistent Hash Based on the Pseudo-Random Sequence

A Novel Technique for Handling Small File Problem of HDFS: Hash Based Archive File (HBAF)

Contact Info

Product

Resources

About