Runtime Object Lifetime Profiler for Latency Sensitive Big Data Applications

Bruno, Rodrigo; Patrício, Duarte; Simão, José; Veiga, Luís; Ferreira, Paulo

doi:10.1145/3302424.3303988

Cited by 27 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior work predicts object lifetime as long or short based on allocation site and precisely matching calling context [11,16] (although Cohn and Singh did use stack data for predictions instead [17]). Current approaches typically store a table of allocation sites, together with a summary of observed persite lifetimes [13]. They either 1) collect lifetime information at runtime, i.e., dynamic pretenuring [16,30] or 2) use profileguided optimization (PGO), collecting lifetimes offline with special instrumentation, analyzing it offline, and then using it in deployment [11].…”

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

“…Table 1 shows recording the calling stack for an allocation can take an order of magnitude longer than the allocation, which is problematic. Solutions include instrumenting the stack prologue and epilogue to keep track of the current stack through a series of bits stored in a register [12,13,29]. However, overheads of this approach are ≈6% and higher, exceeding all the time spent in memory allocation [31].…”

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

“…Overheads. Continuous profiling in deployment is not practical because it adds 6% overhead [13,42], which can be more than memory allocation itself [31].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning-based Memory Allocation for C++ Server Workloads

Maas

Andersen

Isard

et al. 2020

Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Syste

View full text Add to dashboard Cite

Modern C++ servers have memory footprints that vary widely over time, causing persistent heap fragmentation of up to 2× from long-lived objects allocated during peak memory usage. This fragmentation is exacerbated by the use of huge (2MB) pages, a requirement for high performance on large heap sizes. Reducing fragmentation automatically is challenging because C++ memory managers cannot move objects.This paper presents a new approach to huge page fragmentation. It combines modern machine learning techniques with a novel memory manager (Llama) that manages the heap based on object lifetimes and huge pages (divided into blocks and lines). A neural network-based language model predicts lifetime classes using symbolized calling contexts. The model learns context-sensitive per-allocation site lifetimes from previous runs, generalizes over different binary versions, and extrapolates from samples to unobserved calling contexts. Instead of size classes, Llama's heap is organized by lifetime classes that are dynamically adjusted based on observed behavior at a block granularity.Llama reduces memory fragmentation by up to 78% while only using huge pages on several production servers. We address ML-specific questions such as tolerating mispredictions and amortizing expensive predictions across application execution. Although our results focus on memory allocation, the questions we identify apply to other system-level problems with strict latency and resource requirements where machine learning could be applied.

show abstract

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

Section: Lifetime Prediction Challengesmentioning

confidence: 99%

See 1 more Smart Citation

Learning-based Memory Allocation for C++ Server Workloads

Maas

Andersen

Isard

et al. 2020

Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Syste

View full text Add to dashboard Cite

show abstract

“…However, none of those studies provides a characterization on such a wide range of managed applications (from both Dacapo and Renaissance benchmark suites) on top of NUMA machines. Finally, several profiling infrastructures [6,21,28,30] are available for managed runtimes. However, they are either task-specific (ROLP and FJProfiler) or not built for NUMA systems (JProfiler, AntTracks), as well as none of them supports hardware counters utilization.…”

Section: Introductionmentioning

confidence: 99%

You can’t hide you can’t run: a performance assessment of managed applications on a NUMA machine

Papadakis

Zakkak

Foutris

et al. 2020

Proceedings of the 17th International Conference on Managed Programming Languages and Runtimes

View full text Add to dashboard Cite

The ever-growing demand for more memory capacity from applications has always been a challenging factor in computer architecture. The advent of the Non Unified Memory Access (NUMA) architecture has achieved to work around the physical constraints of a single processor by providing more system memory using pools of processors, each with their own memory elements, but with variable access times. However, the efficient exploitation of such computing systems is a non-trivial task for software engineers. We have observed that the performance of more than half of the applications picked from two distinct benchmark suites is negatively affected when running on a NUMA machine, in the absence of manual tuning. This finding motivated us to develop a new profiling tool, so called PerfUtil, to study, characterize and better understand why those benchmarks have sub-optimal performance on NUMA machines. PerfUtil's effectiveness is based on its ability to track numerous events throughout the system at the managed runtime system level, that, ultimately, assists in demystifying NUMA peculiarities and accurately characterize managed applications profiles. CCS CONCEPTS • Software and its engineering → Software design engineering.

show abstract

“…In this paper, rather than reducing an algorithm's complexity (e.g. by decreasing the number of similarity computations), we propose to pursue an orthogonal strategy that is motivated by the system bottlenecks induced by large data volumes: large amounts of data not only stress complex algorithms, they also choke the underlying computation pipelines these algorithms execute on [12].…”

Section: Introductionmentioning

confidence: 99%

Smaller, Faster & Lighter KNN Graph Constructions

Guerraoui

Kermarrec²,

Ruas

et al. 2020

Proceedings of the Web Conference 2020

View full text Add to dashboard Cite

We propose GoldFinger, a new compact and fast-to-compute binary representation of datasets to approximate Jaccard's index. We illustrate the effectiveness of GoldFinger on the emblematic big data problem of K-Nearest-Neighbor (KNN) graph construction and show that GoldFinger can drastically accelerate a large range of existing KNN algorithms with little to no overhead. As a side effect, we also show that the compact representation of the data protects users' privacy for free by providing k-anonymity and l-diversity. Our extensive evaluation of the resulting approach on several realistic datasets shows that our approach delivers speedups of up to 78.9% compared to the use of raw data while only incurring a negligible to moderate loss in terms of KNN quality. To convey the practical value of such a scheme, we apply it to item recommendation and show that the loss in recommendation quality is negligible.

show abstract

Runtime Object Lifetime Profiler for Latency Sensitive Big Data Applications

Cited by 27 publications

References 34 publications

Learning-based Memory Allocation for C++ Server Workloads

Learning-based Memory Allocation for C++ Server Workloads

You can’t hide you can’t run: a performance assessment of managed applications on a NUMA machine

Smaller, Faster & Lighter KNN Graph Constructions

Contact Info

Product

Resources

About