Cycler: Improve Heap Management for Allocation-Intensive Applications with On-the-Fly Object Reuse

Abstract

Tracing garbage collection (GC) identifies dead objects by scanning the object-graph from the root set references. It is proven to be highly efficient and widely used in commercial managed runtime systems for Java, C# and scripting languages. However, tracing GC cannot identify the dead objects before finishing object-graph traversal. Hence the application cannot reuse the space of the dead objects in realtime when the objects are no longer live. Even there are lots of dead objects, the system has to keep allocating new space for new objects until the object-graph is scanned, which has negative impact on both space efficiency, bandwidth efficiency and cache efficiency. The issue becomes serious with allocation-intensive applications. Some prior work tries to improve tracing GC with compilation analysis to identify the dead objects in compilation time. But they can only work with sub-optimal GC algorithm such as mark-sweep, and/or can only discover a small portion of the dead objects. In this paper we propose an algorithm Cycler that can reuse the dead objects in real-time before the object graph scanning. Cycler enhances the existing high-performance tracing GC with reference counting. The novelty of the algorithm is that it does not interfere with the existing optimal tracing GC, so it can effectively improve the overall performance, as long as the runtime overhead of reference counting is well constrained. To control the runtime overhead, Cycler only counts the object references of major types. In this paper, we describe Cycler design and implementation, and also present our evaluations with standard benchmarks.