DeDu: Building a deduplication storage system over cloud computing

Sun, Zhe; Shen, Jun; Yong, Jianming

doi:10.1109/cscwd.2011.5960097

Cited by 21 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Once the initial user changes the original data, the system will set the changed data as a new one, and a new link file will be given to the user. The other users who connect to the original file will not be impacted [24]. Under these conditions, the more users share the same data, the more storage space will be saved.…”

Section: System Mechanismmentioning

confidence: 93%

“…One is used to store mass data, and the other is used to keep the sparse index. On the one hand, there are several secondary storage systems, like Ceph [24], Petal [9], being used as mass data storage systems. On the other hand, there are several database systems such as SQL, Oracle, HBase, and BigTable [8] that can be used as index systems.…”

Section: Storage Systemmentioning

confidence: 99%

See 1 more Smart Citation

A novel approach to data deduplication over the engineering-oriented cloud systems

Sun

Shen

Yong

2013

ICA

Self Cite

View full text Add to dashboard Cite

This paper presents a duplication-less storage system over the engineering-oriented cloud computing platforms. Our deduplication storage system, which manages data and duplication over the cloud system, consists of two major components, a front-end deduplication application and a mass storage system as backend. Hadoop distributed file system (HDFS) is a common distribution file system on the cloud, which is used with Hadoop database (HBase). We use HDFS to build up a mass storage system and employ HBase to build up a fast indexing system. With a deduplication application, a scalable and parallel deduplicated cloud storage system can be effectively built up. We further use VMware to generate a simulated cloud environment. The simulation results demonstrate that our deduplication storage system is sufficiently accurate and efficient for distributed and cooperative data intensive engineering applications Abstract. This paper presents a duplication-less storage system over the engineering-oriented cloud computing platforms. Our deduplication storage system, which manages data and duplication over the cloud system, consists of two major components, a front-end deduplication application and a mass storage system as back-end. Hadoop distributed file system (HDFS) is a common distribution file system on the cloud, which is used with Hadoop database (HBase). We use HDFS to build up a mass storage system and employ HBase to build up a fast indexing system. With a deduplication application, a scalable and parallel deduplicated cloud storage system can be effectively built up. We further use VMware to generate a simulated cloud environment. The simulation results demonstrate that our deduplication storage system is sufficiently accurate and efficient for distributed and cooperative data intensive engineering applications.

show abstract

Section: System Mechanismmentioning

confidence: 93%

Section: Storage Systemmentioning

confidence: 99%

A novel approach to data deduplication over the engineering-oriented cloud systems

Sun

Shen

Yong

2013

ICA

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently Deduplication motivated to the most of researchers due to its popularity. These researchers works on different aspect of deduplication : Semantic attribute based source deduplication, Hadoop based deduplication [7], GPU based deduplication [8], SSD based deduplication [6], deduplication on storage disk [9], signature based deduplication, live deduplication [10], Image based deduplication [11], Scalable deduplication in VM image [12].…”

Section: Existing Methodologies Of Deduplicationmentioning

confidence: 99%

A Survey: Deduplication Ontologies

Patange¹,

Varpe²

2015

IJCA

View full text Add to dashboard Cite

Cloud computing drive out the need of IT based companies to invest in high computing infrastructure and services used by them. In cloud, the data is dwell into set of networked resources that enable data to be accessed via virtual machines. These data centers are located in various parts of the world beyond the control and reach of the user, so there are multiple challenges and security issues that need to be addressed and understood. This review paper aims to analyze and elaborate deduplication issues in cloud computing which is the base for our future roadmap.

show abstract

“…Existing systems and proprietary solutions (such as Dropbox [18] and Google Drive [19]) fails to prevent redundancy of encrypted data in cloud. Another efficient de-duplication system example "DeDu: Deduplication Storage System over Cloud Computing" [21], fails to handle cipher data. Since our proposed system deals with both the issues namely, data integrity audit and secure redundancy prevention mechanism, we prominently focus on reviewing the previous works in these areas.…”

Section: Related Workmentioning

confidence: 99%

Redundancy Prevention and Secure Audit of Encrypted BigData in HDFS Cloud using CloudGaurd+ System

Shevade¹,

Kulkarni²

2018

IJET

View full text Add to dashboard Cite

The recent emerging Cloud Storageand Computing Technology provides a better approach for administrations and service provisioning over the Internet byoffering various computing and storage resources. The key significant and favored cloud service’s is information storage. Therefore, to maintain and conserve the privacy of data owners, data is often encrypted and stored onto the cloud as ciphered data. Nevertheless, a ciphered data introduces some issues for cloud data redundancy, which has been proved to be critical for Bigdata storage and computation in cloudenvironment. Conventional redundancy prevention methods do not work efficiently over ciphered data. Existing systems forciphered data redundancy prevention raises security issues and eventually effects the cloud processing. These existing systems are not capable of efficiently handling data access control and revocations. Therefore aiming to overcome these issues and achieving both redundancy prevention and integrity of cloud data, in this paper we propose a system, namely CloudGaurd+, to prevent redundancy of the stored ciphered data in HDFS Cloud based on authentication provocation and cryptography techniques. CloudGaurd+ incorporates cloud information redundancy prevention with access control. We assess its efficiency and performance based on substantial analysis and computer programming and simulations. The investigation and reenactment comes about demonstrates the unrivaled productivity and adequacy of the CloudGaurd+ system for prospective deployment practically , particularly for Bigdata redundancy prevention in HDFS distributed cloud storage can be achieved.

show abstract

DeDu: Building a deduplication storage system over cloud computing

Cited by 21 publications

References 3 publications

A novel approach to data deduplication over the engineering-oriented cloud systems

A novel approach to data deduplication over the engineering-oriented cloud systems

A Survey: Deduplication Ontologies

Redundancy Prevention and Secure Audit of Encrypted BigData in HDFS Cloud using CloudGaurd+ System

Contact Info

Product

Resources

About