Lifetime-based memory management for distributed data processing systems

Abstract

In-memory caching of intermediate data and eager combining of data in shuffle buffers have been shown to be very effective in minimizing the re-computation and I/O cost in distributed data processing systems like Spark and Flink. However, it has also been widely reported that these techniques would create a large amount of long-living data objects in the heap, which may quickly saturate the garbage collector, especially when handling a large dataset, and hence would limit the scalability of the system. To eliminate this problem, we propose a lifetime-based memory management framework, which, by automatically analyzing the userdefined functions and data types, obtains the expected lifetime of the data objects, and then allocates and releases memory space accordingly to minimize the garbage collection overhead. In particular, we present Deca, a concrete implementation of our proposal on top of Spark, which transparently decomposes and groups objects with similar lifetimes into byte arrays and releases their space altogether when their lifetimes come to an end. An extensive experimental study using both synthetic and real datasets shows that, in comparing to Spark, Deca is able to 1) reduce the garbage collection time by up to 99.9%, 2) to achieve up to 22.7x speed up in terms of execution time in cases without data spilling and 41.6x speedup in cases with data spilling, and 3) to consume up to 46.6% less memory.

An experimental evaluation of garbage collectors on big data applications

“…Therefore, the collector can avoid periodically tracing these objects at each young/full GC. This method is similar to but more lightweight than the lifetime-based memory management proposed in Deca [42], which uses multiple self-defined data containers (byte arrays) to manage the objects with different lifetimes. The key advantage of our proposed approach is that it does not need to determine the number of allocated data containers, merge the data objects into different lifetimes, nor manually reclaim the objects.…”

“…Framework memory management optimization. Researchers have proposed memory configuration tuning strategies [52], regionbased or lifetime-based memory management [33,37,43,45,42] for improving memory utilization and GC optimization. MemTune [52] dynamically adjusts the data cache size and cache policy based on memory usage statistics.…”

“…Facade [45] proposes a compiler and runtime system to bound the number of in-memory data objects, through storing data in an off-heap region and manipulating the data with control interfaces. Deca [42] proposes a lifetime-based memory manager to reduce GC overhead, through analyzing the lifetimes of the data objects in user-defined functions and grouping the objects with similar lifetimes into byte arrays. Garbage collection optimization for big data applications.…”

“…MemTune [52] dynamically tunes the configuration for data cache to optimize the memory utilization. Facade [45], Deca [42], and Broom [37] propose region-based and lifetime-based memory managers to reduce the GC overhead. Recently, Yak [44] and NG2C [32] optimize garbage collection mechanisms, such as heap layout and GC algorithms.…”

NG2C: pretenuring garbage collection with dynamic generations for HotSpot big data applications

“…There are some solutions based on off-heap memory [41,44,46] (i.e., allocating memory for the application outside the GCmanaged heap). While this is an effective approach to allocate and keep data out of the range of the GC (and therefore, reducing object copying), it has several important drawbacks: i) off-heap data needs to be serialized to be saved in off-heap memory, and de-serialized before being used by the application (this obviously has performance overheads); ii) off-heap memory must be explicitly collected by the application developer (which is error prone [11,14] and completely ignores the advantages of running inside Figure 1: Allocation of Objects in Different Generations a memory managed environment); iii) the application must always have objects identifying the data stored in off-heap (these so called header objects are stored in the managed heap therefore stressing the GC).…”