SAM: A Semantic-Aware Multi-tiered Source De-duplication Framework for Cloud Backup

Abstract

Existing de-duplication solutions in cloud backup environment either obtain high compression ratios at the cost of heavy de-duplication overheads in terms of increased latency and reduced throughput, or maintain small de-duplication overheads at the cost of low compression ratios causing high data transmission costs, which results in a large backup window. In this paper, we present SAM, a Semantic-Aware Multitiered source de-duplication framework that first combines the global file-level de-duplication and local chunk-level deduplication, and further exploits file semantics in each stage in the framework, to obtain an optimal tradeoff between the deduplication efficiency and de-duplication overhead and finally achieve a shorter backup window than existing approaches. Our experimental results with real world datasets show that SAM not only has a higher de-duplication efficiency/overhead ratio than existing solutions, but also shortens the backup window by an average of 38.7%.

SecDep: A user-aware efficient fine-grained secure deduplication scheme with multi-level key management

“…According to the results shown in Table II, we obtain two key observations. [21] and Microsoft's study [2], have similar observations. Meanwhile, there are remaining substantial duplicate chunks inside users.…”

“…One-set was collected from 11 graduate students of a research group and was reported by Xia et al [32]. Inc-set was collected from initial full backups and subsequent incremental backups of 6 members of a university research group and was reported by Tan et al [21]. Full-set consists of 380 full backups of 19 researchers' PC and is reported by Xing et al [33].…”

Optimization Techniques To Record Deduplication

“…Particularly, there is communication between client software and the backup server to check for the presence of files or blocks (Harnik et al, 2010). Two well-known source de-duplication methods, source local chunk-level deduplication (Tan et al, 2010) and source global chunklevel de-duplication have been proposed in the past to address the above mentioned problem by erasing the redundant data chunks before transfering them to the remote backup destination.…”

Design and Implementation of File Deduplication Framework on HDFS

“…Chun-Ho Ng's LiveDFS focused on the deduplication of VM image in the Open-Source cloud and achieved at least 2 International Journal of Distributed Sensor Networks 40% of storage saving for VM images storage with reasonable performance [11]. SAM [12] is a semantic-aware multitiered source deduplication framework for cloud backup system. It gets a high deduplication ratio which is as better as global chunk-based deduplication and very low overhead than that of global chunk-based deduplication.…”