Hash challenges: Stretching the limits of compare-by-hash in distributed data deduplication

Barreto, João; Veiga, Luís; Ferreira, Paulo

doi:10.1016/j.ipl.2012.01.012

Cited by 11 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…JoaoBarreto [12] proposed an alternative technique to CBH called Hash Challenges(HCs). Using HCs, he tried to find the redundant chunks by using hash fragments instead of their complete hashes.…”

Section: Related Workmentioning

confidence: 99%

WITHDRAWN: RETRACTED: Optimizing the performance of Image Deduplication system for effective storage in cloud using Enhanced Prefix Hash Tree

Sujatha

RetnaRaj

2021

Preprint

View full text Add to dashboard Cite

Today the world is highly technological and well connected. Data grows massively every day, exponentially with time. Data handled by individuals or organizations are increasing day by day. Even though it shows positive signs, the main problem is, they do not have enough space to store their growing collection of data. With the increasing development of cloud storage services, individuals and organizations started uploading their data in the cloud storage space. It allows the client to store all types of digital data. It provides its service in three different models as Public cloud, Private cloud, and Hybrid cloud. The public cloud storage space can be shared among many users. Since many users sharing the cloud storage space to store their data, duplicate data contents being stored cannot be avoided. It may lead to poor utilization of storage space. Especially in the case of Image and Video, the wastage of memory due to duplicate copy is extremely high. An essential storage optimization technique called “Deduplication” can be employed to overcome this problem. But it takes time to find duplicate copies in the cloud. In a hash-based deduplication system, additional memory space is required to store the hash value of the digital data. To reduce the overhead caused by this process, we propose an optimization technique to optimize the performance of the text and Image deduplication system using an Enhanced Prefix Hash Tree.

show abstract

Section: Related Workmentioning

confidence: 99%

WITHDRAWN: RETRACTED: Optimizing the performance of Image Deduplication system for effective storage in cloud using Enhanced Prefix Hash Tree

Sujatha

RetnaRaj

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For example SHA-1 (20 byte) and MD5 (16 byte) hash function is used. Joao Barreto [11] used SHA-1 algorithm to compute hashes for the chunks. Though SHA-1 avoids hash collision, it is more CPU intensive than MD5 hash algorithm.…”

Section: Related Workmentioning

confidence: 99%

“…Compare-by-Hash method to reduce lookup latency and communication overhead across the network [11]. The Deduplication is performed by comparing k-byte prefix of each hash.…”

Section: Barreto Proposed Hash Challenges [Hc] Which Is Similar Tomentioning

confidence: 99%

Efficient fingerprint lookup using Prefix Indexing Tablet

Priyadharshini

Angelina

Sundarakantham

et al. 2014

2014 International Conference on Recent Trends in Information Technology

View full text Add to dashboard Cite

Backups protect the file systems from disk or other hardware failures, software errors that may corrupt the file system and natural disasters. However, a single file may be present as multiple copies in the file system. Hence searching time to find the redundant data and to eliminate them is high. In addition to this, redundant data consumes more space in storage systems. Data de-duplication techniques are used to address these issues. Fingerprint lookup is a key ingredient for efficient de-duplication. This paper proposes an efficient Fingerprint lookup technique called Prefix Indexing Tablets in which the fingerprint lookup is performed only on necessary tablets. Further to reduce the fingerprint lookup delay, only the prefix of the fingerprint is considered. Experimentation on standard datasets show that the lookup latency of the proposed deduplication method is reduced by 62% and the running time is improved.

show abstract

“…The very big challenge in handling the image deduplication process 1,17–20 in cloud storage space is time and space complexity. The time required to search the existence of duplicate copy and memory required to store all these hash values is tremendously high which often leads to additional overhead for the cloud users.…”

Section: Introductionmentioning

confidence: 99%

An efficient enhanced prefix hash tree model for optimizing the storage and image deduplication in cloud

Sujatha

Raj

2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary The popularity of the cloud storage space mainly attracted organizations to store their data in them. Therefore, the avoidance of duplicate data contents is unavoidable and several users share the cloud storage space for data storage, and sometimes this makes higher storage space utilization. Because of the extremely high duplicate copy, memory wastage arises in the case of multimedia data. Identifying the final duplicate copies in the cloud takes more time. To overcome this problem, we employ a significant storage optimization model for deduplication. The digital data hash value is stored by requiring an additional memory space. This study proposed an enhanced prefix hash tree (EPHT) method to optimize the image and text deduplication system to reduce the overhead caused by this procedure. The efficiency of the proposed approach is compared with the interpolation search technique using different levels of tree height (2, 4, 2, 8, 16) in terms of space and time complexity. The proposed EPHT technique shows improvements in terms of speed and space complexity when the number of levels in the EPHT increases.

show abstract

Hash challenges: Stretching the limits of compare-by-hash in distributed data deduplication

Cited by 11 publications

References 11 publications

WITHDRAWN: RETRACTED: Optimizing the performance of Image Deduplication system for effective storage in cloud using Enhanced Prefix Hash Tree

WITHDRAWN: RETRACTED: Optimizing the performance of Image Deduplication system for effective storage in cloud using Enhanced Prefix Hash Tree

Efficient fingerprint lookup using Prefix Indexing Tablet

An efficient enhanced prefix hash tree model for optimizing the storage and image deduplication in cloud

Contact Info

Product

Resources

About