Fast restore of checkpointed memory using working set estimation

Zhang, Irene; Garthwaite, Alex; Baskakov, Yury; Barr, Kenneth C.

doi:10.1145/2007477.1952695

Cited by 27 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Snowflock extends the idea of VM cloning to instantiating VMs across entire clusters, relying on lazy guest memory loading to avoid large transfers of guest memory contents across network [38]. To minimize the time spent in serving the series of lazy page faults during guest memory loading, the researchers explore a variety of working set prediction and prefetching techniques [35,[66][67][68]. These techniques rely on profiling of the memory accesses after the moment a checkpoint was taken and inspecting the locality characteristics of the guest OS' virtual address space.…”

Section: Virtual Machine Snapshotsmentioning

confidence: 99%

Benchmarking, analysis, and optimization of serverless function snapshots

Ustiugov

Petrov

Kogias

et al. 2021

Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

Serverless computing has seen rapid adoption due to its high scalability and flexible, pay-as-you-go billing model. In serverless, developers structure their services as a collection of functions, sporadically invoked by various events like clicks. High inter-arrival time variability of function invocations motivates the providers to start new function instances upon each invocation, leading to significant cold-start delays that degrade user experience. To reduce cold-start latency, the industry has turned to snapshotting, whereby an image of a fully-booted function is stored on disk, enabling a faster invocation compared to booting a function from scratch.This work introduces vHive, an open-source framework for serverless experimentation with the goal of enabling researchers to study and innovate across the entire serverless stack. Using vHive, we characterize a state-of-the-art snapshot-based serverless infrastructure, based on industry-leading Containerd orchestration framework and Firecracker hypervisor technologies. We find that the execution time of a function started from a snapshot is 95% higher, on average, than when the same function is memoryresident. We show that the high latency is attributable to frequent page faults as the function's state is brought from disk into guest memory one page at a time. Our analysis further reveals that functions access the same stable working set of pages across different invocations of the same function. By leveraging this insight, we build REAP, a light-weight software mechanism for serverless hosts that records functions' stable working set of guest memory pages and proactively prefetches it from disk into memory. Compared to baseline snapshotting, REAP slashes the cold-start delays by 3.7×, on average. CCS CONCEPTS• Computer systems organization → Cloud computing; • Information systems → Computing platforms; Data centers; • Software and its engineering → n-tier architectures.

show abstract

Section: Virtual Machine Snapshotsmentioning

confidence: 99%

Benchmarking, analysis, and optimization of serverless function snapshots

Ustiugov

Petrov

Kogias

et al. 2021

Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

show abstract

“…Some perform prefetching by predicting a program's future accesses based on earlier memory accesses, often from within the same execution [4,17,22,42]. Others perform targeted prefetching based on measurements of a program's working sets from prior executions [47,52]. Some prior works have even used machine learning to predict future memory accesses based on prior ones [21,26,41].…”

Section: Related Workmentioning

confidence: 99%

3PO: Programmed Far-Memory Prefetching for Oblivious Applications

Branner-Augmon¹,

Galstyan²,

Kumar³

et al. 2022

Preprint

View full text Add to dashboard Cite

Using memory located on remote machines, or far memory, as a swap space is a promising approach to meet the increasing memory demands of modern datacenter applications. Operating systems have long relied on prefetchers to mask the increased latency of fetching pages from swap space to main memory. Unfortunately, with traditional prefetching heuristics, performance still degrades when applications use far memory. In this paper we propose a new prefetching technique for far-memory applications. We focus our efforts on memory-intensive, oblivious applications whose memory access patterns are independent of their inputs, such as matrix multiplication. For this class of applications we observe that we can perfectly prefetch pages without relying on heuristics. However, prefetching perfectly without requiring significant application modifications is challenging.In this paper we describe the design and implementation of 3PO, a system that provides pre-planned prefetching for general oblivious applications. We demonstrate that 3PO can accelerate applications, e.g., running them 30-150% faster than with Linux's prefetcher with 20% local memory. We also use 3PO to understand the fundamental software overheads of prefetching in a paging-based system, and the minimum performance penalty that they impose when we run applications under constrained local memory.

show abstract

“…Thus, we use our staged approach ( §6) to find a good approximation. Some systems use observations of past memory accesses or past working sets (e.g., from prior invocations of a program) to perform targeted prefetching [33,35,56,77,92] and approx-imate Belady's algorithm (MIN) [72]. SC's obliviousness and our memory programming approach allow MAGE to compute the memory access pattern without first running the program, and then apply these techniques using the access pattern itself.…”

Section: Related Workmentioning

confidence: 99%

MAGE: Nearly Zero-Cost Virtual Memory for Secure Computation

Kumar

Culler

Popa

2021

Preprint

View full text Add to dashboard Cite

Secure Computation (SC) is a family of cryptographic primitives for computing on encrypted data in single-party and multi-party settings. SC is being increasingly adopted by industry for a variety of applications. A significant obstacle to using SC for practical applications is the memory overhead of the underlying cryptography. We develop MAGE, an execution engine for SC that efficiently runs SC computations that do not fit in memory. We observe that, due to their intended security guarantees, SC schemes are inherently oblivioustheir memory access patterns are independent of the input data. Using this property, MAGE calculates the memory access pattern ahead of time and uses it to produce a memory management plan. This formulation of memory management, which we call memory programming, is a generalization of paging that allows MAGE to provide a highly efficient virtual memory abstraction for SC. MAGE outperforms the OS virtual memory system by up to an order of magnitude, and in many cases, runs SC computations that do not fit in memory at nearly the same speed as if the underlying machines had unbounded physical memory to fit the entire computation.

show abstract

Fast restore of checkpointed memory using working set estimation

Cited by 27 publications

References 11 publications

Benchmarking, analysis, and optimization of serverless function snapshots

Benchmarking, analysis, and optimization of serverless function snapshots

3PO: Programmed Far-Memory Prefetching for Oblivious Applications

MAGE: Nearly Zero-Cost Virtual Memory for Secure Computation

Contact Info

Product

Resources

About