A Lookahead Read Cache: Improving Read Performance for Deduplication Backup Storage

Park, Dongchul; Fan, Ziqi; Nam, Young Jin; Du, David H. C.

doi:10.1007/s11390-017-1680-8

Cited by 16 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pre-fetching and look-ahead schemes: In order to provide efficient caching for the deduplication enabled data, several prior works that adopts look-ahead and pre-fetching schemes were implemented [20], [21], [22], [24], [48]. However, these approaches are limited to localized (single-node) deduplication platforms.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Accelerating ML/DL Applications with Hierarchical Caching on Deduplication Storage Clusters

Hamandawana

Khan

Kim

et al. 2021

IEEE Trans. Big Data

View full text Add to dashboard Cite

Large scale machine learning (ML) and deep learning (DL) platforms face challenges when integrated with deduplication enabled storage clusters. In the quest to achieve smart and efficient storage utilization, removal of duplicate data introduces bottlenecks, since deduplication alters the I/O transaction layout of the storage system. Therefore, it is critical to address such deduplication overhead for acceleration of ML/DL computation in deduplication storage. Existing state of the art ML/DL storage solutions such as Alluxio and AutoCache adopt non deduplication-aware caching mechanisms, which lacks the much needed performance boost when adopted in deduplication enabled ML/DL clusters. In this paper, we introduce Redup, which eliminates the performance drop caused by enabling deduplication in ML/DL storage clusters. At the core, is a Redup Caching Manager (RDCM), composed of a 2-tier deduplication layoutaware caching mechanism. The RDCM provides an abstraction of the underlying deduplication storage layout to ML/DL applications and provisions a decoupled acceleration of object reconstruction during ML/DL read operations. Our Redup evaluation shows negligible performance drop in ML/DL training performances as compared to a cluster without deduplication, whilst significantly outperforming Alluxio and AutoCache in terms of various performance metrics.

show abstract

Section: Related Workmentioning

confidence: 99%

“…A lot of prior works have employed optimized pre-fetching and look-ahead caching schemes [20], [21], [22], [24], which amortize the read I/O penalty in deduplicated storage. However, these works are optimized for local node deduplication and not at cluster scale level.…”

Section: Introductionmentioning

confidence: 99%

Accelerating ML/DL Applications with Hierarchical Caching on Deduplication Storage Clusters

Hamandawana

Khan

Kim

et al. 2021

IEEE Trans. Big Data

View full text Add to dashboard Cite

show abstract

“…Unlike the deduplication phase that searches the metadata (i.e., fingerprint indexes), the restore phase accesses the real data of the chunks according to the recipe. The restore performance suffers from the chunk fragmentation problem [8,9,13,16,20,30], i.e., the chunks of the same data stream are scattered into various containers, causing frequent disk accesses during the recoveries. The chunk fragmentation problem comes during the deduplication phase.…”

Section: Chunk Fragmentation Problemmentioning

confidence: 99%

“…The data grow exponentially in the widely used applications, such as IoT embeddings, artificial intelligence and cloud computing, which require efficient and large-scale storage capacities [7,17,18]. To save space and improve storage efficiency, data deduplication [31,41] becomes an efficient middleware to eliminate the duplicate data, and has been widely used in current storage systems [11, 24-26, 32, 39], especially for storage backup systems [14,19,30].…”

Section: Introductionmentioning

confidence: 99%

Improving the Restore Performance via Physical-Locality Middleware for Backup Systems

Hua

Cao

et al. 2020

Proceedings of the 21st International Middleware Conference

View full text Add to dashboard Cite

“…Usually, the dimension of the data is unified by the translation to standard deviation transformation and the shift to range transformation [37][38][39][40][41], and the fuzzy matrix in fuzzy clustering is obtained. If the degree of similarity which is the coefficient of similarity [42][43][44][45][46][47][48].…”

Section: Fuzzy Cluster Analysis Modelmentioning

confidence: 99%

Location Selection of Logistics Center in e-Commerce Network Environments

Chen¹

2017

AJNNA

View full text Add to dashboard Cite

Abstract:The site selection of logistics center is a very complicated and enormous system problem. Each site selection method and model is based on a certain premise and hypothesis. The site selection decision of distribution center has an important influence on the whole logistics system operation. A good location of urban logistics sites is important for optimizing the logistics network, and improving the urban traffic conditions, and level of logistics services. Motivated by this, in this paper, based on the research of sixteen cities in southeastern China and neural network algorithms, we proposed a logistics center location selection algorithm. Our method considers the six important concepts reflecting the performance index of the city logistics, such as city location quotient, market prosperity degree, proportion of freight volume, urban centricity, per capita gross domestic product (GDP), and population size. Our method conducts the nested fuzzy analytic hierarchy process (AHP), and then investigates the total ranking of the single order hierarchy to select suitable logistics centers.

show abstract

A Lookahead Read Cache: Improving Read Performance for Deduplication Backup Storage

Cited by 16 publications

References 18 publications

Accelerating ML/DL Applications with Hierarchical Caching on Deduplication Storage Clusters

Accelerating ML/DL Applications with Hierarchical Caching on Deduplication Storage Clusters

Improving the Restore Performance via Physical-Locality Middleware for Backup Systems

Location Selection of Logistics Center in e-Commerce Network Environments

Contact Info

Product

Resources

About