Efficient context sensitivity for dynamic analyses via calling context uptrees and customized memory management

Huang, Juan; Bond, Michael D.

doi:10.1145/2509136.2509510

Cited by 12 publications

(7 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 shows recording the calling stack for an allocation can take an order of magnitude longer than the allocation, which is problematic. Solutions include instrumenting the stack prologue and epilogue to keep track of the current stack through a series of bits stored in a register [12,13,29]. However, overheads of this approach are ≈6% and higher, exceeding all the time spent in memory allocation [31].…”

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

Learning-based Memory Allocation for C++ Server Workloads

Maas

Andersen

Isard

et al. 2020

Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Syste

View full text Add to dashboard Cite

Modern C++ servers have memory footprints that vary widely over time, causing persistent heap fragmentation of up to 2× from long-lived objects allocated during peak memory usage. This fragmentation is exacerbated by the use of huge (2MB) pages, a requirement for high performance on large heap sizes. Reducing fragmentation automatically is challenging because C++ memory managers cannot move objects.This paper presents a new approach to huge page fragmentation. It combines modern machine learning techniques with a novel memory manager (Llama) that manages the heap based on object lifetimes and huge pages (divided into blocks and lines). A neural network-based language model predicts lifetime classes using symbolized calling contexts. The model learns context-sensitive per-allocation site lifetimes from previous runs, generalizes over different binary versions, and extrapolates from samples to unobserved calling contexts. Instead of size classes, Llama's heap is organized by lifetime classes that are dynamically adjusted based on observed behavior at a block granularity.Llama reduces memory fragmentation by up to 78% while only using huge pages on several production servers. We address ML-specific questions such as tolerating mispredictions and amortizing expensive predictions across application execution. Although our results focus on memory allocation, the questions we identify apply to other system-level problems with strict latency and resource requirements where machine learning could be applied.

show abstract

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

Learning-based Memory Allocation for C++ Server Workloads

Maas

Andersen

Isard

et al. 2020

Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Syste

View full text Add to dashboard Cite

show abstract

“…During profiling, we label objects with their allocation site at allocation time. We associate each allocation site with a unique identifier which the compiler creates when it first encounters each new bytecode during profiling, following prior work [28]. The compiler generates an allocation sequence that stores this identifier in the header of each object.…”

Section: Profilingmentioning

confidence: 99%

Crystal Gazer

Akram

Sartor

McKinley

et al. 2019

Proc. ACM Meas. Anal. Comput. Syst.

View full text Add to dashboard Cite

Non-volatile memories (NVM) offer greater capacity than DRAM but suffer from high latency and low write endurance. Hybrid memories combine DRAM and NVM to form scalable memory systems with the promise of high capacity, low energy consumption, and high endurance. Automatically managing hybrid NVM-DRAM memories to achieve their promise without changing user applications or their programming models remains an open question. This paper uses garbage collection in managed languages to exploit NVM capacity while preventing NVM wear out in hybrid memories with no changes to the programming model.We introduce profile-driven write-rationing garbage collection. Allocation sites that produce frequently written objects are predicted based on previous program executions. Objects are initially allocated in a DRAM nursery space. The collector copies surviving nursery objects from highly written sites to a mature DRAM space and read-mostly objects to a mature NVM space. Write-intensity prediction for 15 Java benchmarks accurately places objects in the correct space, eliminating expensive object monitoring from prior write-rationing garbage collectors. Furthermore, our technique exposes a Pareto tradeoff between DRAM usage and NVM lifetime, unlike prior work. Experimental results on NUMA hardware that emulates hybrid NVM-DRAM memory demonstrates that profile-driven write-rationing garbage collection reduces the number of writes to NVM compared to prior work to extend its lifetime, maximizes the use of NVM for its capacity, and achieves good performance. S. Akram et al.worrisome as emerging applications have an insatiable desire for memory. Expecting further DRAM supply shortages, Facebook is already experimenting with hybrid DRAM and non-volatile memory (NVM) systems [23].Production NVM uses phase-change memory (PCM). PCM offers high capacity, low idle power, and non-volatility, while being byte addressable, unlike some NVM. It however suffers two major drawbacks over DRAM: long latency and limited write endurance [16,37,38]. New materials may eventually bridge the latency gap [30,42]. However, endurance will remain a problem because PCM writes change the material form [16]. Although prior hardware and software approaches improve PCM lifetime, an endurance gap remains [3,37,38,48].Memory system analysts and others report that Intel's Optane SSD, a commercially available NVM with 375 GB of capacity, can sustain a write rate of up to 10 TB/day, i.e., 140 MB/s without wearing out within the 3 year warranty period [23,26]. Our experimental results (using an optimistic PCM latency) show that 13 out of our 15 modern Java applications exceed this write rate when using a PCM-only main memory. In practice, memory systems will therefore need hybrid DRAM and PCM memories. Similar products with a smaller capacity (up to 32 GB) only support write rates of up to 1.5 MB/s [26]. All of our Java applications exceed this write rate even using state-of-the-art hybrid memory management.Including PCM into the main memory system requires solutions to t...

show abstract

“…In a recent work [30], a novel data structure is proposed to avoid the node lookup operation in dynamic bug detectors. A new node is instead allocated for each context, and the costs of allocations are mitigated by extending the garbage collector not only to collect unused node but also to merge duplicate ones lazily.…”

Section: Related Workmentioning

confidence: 99%

Mining hot calling contexts in small space

D’Elia¹,

Demetrescu²,

Finocchi³

2011

Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Calling context trees (CCTs) associate performance metrics with paths through a program's call graph, providing valuable information for program understanding and performance analysis. Although CCTs are typically much smaller than call trees, in real applications they might easily consist of tens of millions of distinct calling contexts: this sheer size makes them difficult to analyze and might hurt execution times due to poor access locality. For performance analysis, accurately collecting information about hot calling contexts may be more useful than constructing an entire CCT that includes millions of uninteresting paths. As we show for a variety of prominent Linux applications, the distribution of calling context frequencies is typically very skewed. In this paper we show how to exploit this property to reduce the CCT size considerably.We introduce a novel run-time data structure, called Hot Calling Context Tree (HCCT), that offers an additional intermediate point in the spectrum of data structures for representing interprocedural control flow. The HCCT is a subtree of the CCT that includes only hot nodes and their ancestors. We show how to compute the HCCT without storing the exact frequency of all calling contexts, by using fast and space-efficient algorithms for mining frequent items in data streams. With this approach, we can distinguish between hot and cold contexts on the fly, while obtaining very accurate frequency counts. We show both theoretically and experimentally that the HCCT achieves a similar precision as the CCT in a much smaller space, roughly proportional to the number of distinct hot contexts: this is typically several orders of magnitude smaller than the total number of calling contexts encountered during a program's execution. Our space-efficient approach can be effectively combined with previous context-sensitive profiling techniques, such as sampling and bursting.

show abstract

Efficient context sensitivity for dynamic analyses via calling context uptrees and customized memory management

Cited by 12 publications

References 60 publications

Learning-based Memory Allocation for C++ Server Workloads

Learning-based Memory Allocation for C++ Server Workloads

Crystal Gazer

Mining hot calling contexts in small space

Contact Info

Product

Resources

About