Rambrain - a library for virtually extending physical memory

Abstract

We introduce Rambrain, a user space C++ library that manages memory consumption of data-intense applications. Using Rambrain one can overcommit memory beyond the size of physical memory present in the system. While there exist other more advanced techniques to solve this problem, Rambrain focusses on saving development time by providing a fast, general and easy-to-use solution. Rambrain takes care of temporarily swapping out data to disk and can handle multiples of the physical memory size present. Rambrain is thread-safe, OpenMP and MPI compatible and supports asynchronous IO. The library is designed to require minimal changes to existing programs and pose only a small overhead.

SPH to Grid: a new integral conserving method

“…This results already in 4GB of data just for the three dimensional grid. There are possibilities like outof-core computing to approach this problem, for example the memory management library Rambrain (Imgrund and Arth, 2017) which is actually used by the tool SPHMapper presented in section 5.2. However, this only allows to the user to create a huge grid and somehow fit it into memory without reducing the resulting requirements of CPU time.…”

“…As computing power grows and simulation data sets become increasingly bigger over time the main requirement for SPHMapper is to be able to handle data sets which cannot be kept in main memory all at once. To approach this issue it features the use of Rambrain (Imgrund and Arth, 2017), an automatic memory manager which utilises second storage capacity to overextend memory.…”