Liquid: A Scalable Deduplication File System for Virtual Machine Images

Zhao, Xingang; Zhang, Yang; Wu, Yongwei; Chen, Kang; Jiang, Jinlei; Li, Keqin

doi:10.1109/tpds.2013.173

Cited by 46 publications

(24 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extrapolating these storage requirements shows that Squirrel can scale to thousands of VMI caches with modest disk and memory requirements on current or near-future compute node hardware. Further, we show that, with proper parameter tuning, booting from a deduplicated and compressed file system can be as fast as a normal file system, despite earlier reports [26,44].…”

Section: Introductionmentioning

confidence: 72%

“…When decreasing the block size 3 , we can see two conflicting trends: Deduplication ratio starts increasing, and gzip's compression ratio starts decreasing. As numerous VMI deduplication studies [18,19,26,44] have pointed out, the reasons for higher deduplication ratio with smaller block sizes are (1.) the fact that small differences 3 In this paper, we study the trends when making block size smaller.…”

Section: Compression Efficiencymentioning

confidence: 99%

“…Compared to Squirrel, Glance API servers cannot cache many multi-GB VMIs. The Liquid file system [44] is designed for VMI distribution and has an architecture similar to Squirrel. Liquid keeps a cache of deduplicated VMI contents on each compute node.…”

Section: Storing Vmi Contents On Compute Nodesmentioning

confidence: 99%

See 2 more Smart Citations

Squirrel

Razavi¹,

Ion²,

Kielmann³

2014

Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

In IaaS clouds, virtual machines are booted on demand from user-provided disk images. Both the number of virtual machine images (VMIs) and their large size (GBs), challenge storage and network transfer solutions, and lead to perceivably slow VM startup times. In previous work, we proposed using small VMI caches (O(100 MB)) that contain those parts of a VMI that are actually needed for booting. Here, we present Squirrel, a fully replicated storage architecture that exploits deduplication, compression, and snapshots from the ZFS file system, and lets us keep large quantities of VMI caches on all compute nodes of a data center with modest storage requirements. (Much like rodents cache precious food in many distributed places.) Our evaluation shows that we can store VMI caches for all 600+ community images of Windows Azure, worth 16.4 TB of raw data, within 10 GB of disk space and 60 MB of main memory on each compute node of our DAS-4 cluster. Extrapolation to several thousands of images predicts the scalability of our approach.

show abstract

Section: Introductionmentioning

confidence: 72%

Section: Compression Efficiencymentioning

confidence: 99%

See 1 more Smart Citation

Squirrel

Razavi¹,

Ion²,

Kielmann³

2014

Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…Zhao et al [31] proposed a scalable VMI file system called Liquid that enables large scale VM deployment through a fixed size block level deduplication, resulting in a low storage consumption. The system also improves the I/O performance with a peer-to-peer networked VMI sharing and distribution.…”

Section: Related Workmentioning

confidence: 99%

Semantics-Aware Virtual Machine Image Management in IaaS Clouds

Saurabh

Remmers

Kimovski

et al. 2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

Infrastructure-as-a-service (IaaS) Clouds concurrently accommodate diverse sets of user requests, requiring an efficient strategy for storing and retrieving virtual machine images (VMIs) at a large scale. The VMI storage management require dealing with multiple VMIs, typically in the magnitude of gigabytes, which entails VMI sprawl issues hindering the elastic resource management and provisioning. Nevertheless, existing techniques to facilitate VMI management overlook VMI semantics (i.e at the level of base image and software packages) with either restricted possibility to identify and extract reusable functionalities or with higher VMI publish and retrieval overheads. In this paper, we design, implement and evaluate Expelliarmus, a novel VMI management system that helps to minimize storage, publish and retrieval overheads. To achieve this goal, Expelliarmus incorporates three complementary features. First, it makes use of VMIs modelled as semantic graphs to expedite the similarity computation between multiple VMIs. Second, Expelliarmus provides a semantic aware VMI decomposition and base image selection to extract and store non-redundant base image and software packages. Third, Expelliarmus can also assemble VMIs based on the required software packages upon user request. We evaluate Expelliarmus through a representative set of synthetic Cloud VMIs on the real test-bed. Experimental results show that our semantic-centric approach is able to optimize repository size by 2.2 − 16 times compared to state-ofthe-art systems (e.g. IBM's Mirage and Hemera) with significant VMI publish and retrieval performance improvement.

show abstract

“…Liquid [33] and similar systems (e.g., [13,19,32]) are designed for scalable VM image distribution. Squirrel's cVolumes [23] persistently cache all the blocks needed for starting different VMs by capturing their boot working sets [24] in a compressed ZFS file system.…”

Section: Scalable Transfer/caching Of Vm Imagesmentioning

confidence: 99%

Scaling VM Deployment in an Open Source Cloud Stack

Razavi

Costache

Gardiman

et al. 2015

Proceedings of the 6th Workshop on Scientific Cloud Computing

View full text Add to dashboard Cite

Interactive High Performance Computing (HPC) workloads take advantage of the elasticity of clouds to scale their computation based on user demand by dynamically provisioning virtual machines during their runtime. As in this case users require the results of their computation in a short time, the time to start the provisioned virtual instances becomes crucial. In this paper we study the deployment scalability of OpenNebula, an open-source cloud stack, with respect to these workloads. We describe our efforts for tuning the infrastructure's and OpenNebula's configuration as well as solving scalability issues in its implementation. After tuning both infrastructure and cloud stack, the simultaneous deployment of 512 VMs improved by 5.9× on average, from 615 to 104 seconds, and after optimizing the implementation, the deployment time improved by 12× on average, to 53.54 seconds. These results suggest two possible improvement opportunities that can be useful for both cloud developers and scientific users deploying a cloud stack to avoid such scalability issues in the future. First, the tuning process of a cloud stack can be improved through automatic tools that adapt the configuration to the workload and infrastructure characteristics. Second, the code scalability issues can be avoided through a testing infrastructure that supports large scale emulation.

show abstract

Liquid: A Scalable Deduplication File System for Virtual Machine Images

Cited by 46 publications

References 9 publications

Squirrel

Squirrel

Semantics-Aware Virtual Machine Image Management in IaaS Clouds

Scaling VM Deployment in an Open Source Cloud Stack

Contact Info

Product

Resources

About