Persistence for the masses: RRB-vectors in a systems language

Abstract

Relaxed Radix Balanced Trees (RRB-Trees) is one of the latest members in a family of persistent tree based data-structures that combine wide branching factors with simple and relatively flat structures. Like the battletested immutable sequences of Clojure and Scala, they have effectively constant lookup and updates, good cache utilization, but also logarithmic concatenation and slicing. Our goal is to bring the benefits of functional data structures to the discipline of systems programming via generic yet efficient immutable vectors supporting transient batch updates. We describe a C++ implementation that can be integrated in the runtime of higher level languages with a C core (Lisps like Guile or Racket, but also Python or Ruby), thus widening the access to these persistent data structures. In this work we propose (1) an Embedding RRB-Tree (ERRB-Tree) data structure that efficiently stores arbitrary unboxed types, (2) a technique for implementing tree operations independent of optimizations for a more compact representation of the tree, (3) a policy-based design to support multiple memory management and reclamation mechanisms (including automatic garbage collection and reference counting), (4) a model of transience based on move-semantics and reference counting, and (5) a definition of transience for confluent meld operations. Combining these techniques a performance comparable to that of mutable arrays can be achieved in many situations, while using the data structure in a functional way. CCS Concepts: • Software and its engineering → Data types and structures; Functional languages; • General and reference → Performance;

Kindly bent to free us

“…Recently introduced persistent data structures permit transient mutations where non-linear uses lead to degraded performance [Conchon and Filliâtre 2007] or to dynamic and static checks [Puente 2017]. In particular, persistent Hash-Array-Mapped-Tries (HAMT) have been used with similar APIs in several non-pure functional languages (OCaml, Clojure, .…”

“…While presented for copy-on-write arrays, the API can easily be adapted to other persistent data structures with transient mutability such as Relaxed-Radix Balance Vectors (RRB) [Puente 2017] or persistent HAMTs [Bagwell 2001; Hickey 2017] to provide a convenient programming style without compromising performance.…”

To-many or to-one? all-in-one! efficient purely functional multi-maps with type-heterogeneous hash-tries

“…Instead we need a different representation. Similar to the changes made for more recent immutable sequence data structures [19,26] and binary search trees [21] we need to break out of the original hash-trie design and add some initial overhead to achieve what we want: a leaner and faster implementation of multi-maps and likewise data structure designs (that require fine-grained type-heterogeneity).…”

“…In line with recent efforts of optimizing generic general purpose collections [10,19,24,26], we aim to improve collection data structures towards memory-intensive applications, which is a preliminary for processing larger data sets that fit into main memory.…”

Mod

“…To reduce the memory overhead introduced by shadow paging, our datastructures use structural sharing optimizations found in purely functional datastructures [13,36,38,41,43]. With these optimizations, the updated shadow is built out of the unmodified data of the original datastructure plus modest new and updated state.…”