You can have it all: abstraction and good cache performance

Abstract

On current architectures, the optimisation of an application's performance often involves data being stored according to access affinity -what is accessed together should be stored together, rather than logical affinity -what belongs together logically stays together. Such low level techniques lead to faster, but more error prone code, and end up tangling the program's logic with low-level data layout details.Our vision, which we call SHAPES-Safe, High-level, Abstractions for oPtimisation of mEmory cacheS -is that the layout of a data structure should be defined only once, upon instantiation, and the remainder of the code should be layout agnostic. This enables performance improvements while also guaranteeing memory safety, and supports the separation of program logic from low level concerns. In this paper we investigate how this vision can be supported by extending a programming language.We describe the core language features supporting this vision: classes can be customized to support different layouts, and layout information is carried around in types; the remaining source code is layout-unaware and the compiler emits layout-aware code. We then discuss our SHAPES implementation through a prototype library, which we also used for preliminary evaluations. Finally, we discuss how the core could be expanded so as to deliver SHAPES's full potential: the incorporation of compacting garbage collection, * Work done while at Uppsala University.

Reshape your layouts, not your programs: A safe language extension for better cache locality

“…Considering our objective to have SHAPES support managed languages, it is natural to demand interoperability with existing garbage collectors. For a discussion on movement and compaction in pools, see [16].…”

Extending SHAPES for SIMD Architectures

“…Shapes To address locality and AoS-SoA transformation, we have in the past proposed SHAPES [1], which is a set of abstract layout annotations for managed languages to enable memory optimisations under abstract memory. SHAPES leverages the following concepts: pools and layouts.…”

“…The semantics of SIMD memory instructions in most architectures require data to be laid out in a contiguous fashion in memory. That is, there are no memory scatter-gather operations available most of the time 1 .…”

“…Changing the representation of the data to Struct of Arrays (SoA)-so that vector coordinates are stored in their own subarrays, as depicted in the bottom of Figure 1-enables the iteration over 4 vector pairs at a time. 1 Even when scatter-gather instructions are available (e.g. on AVX), they will suffer multiple cache misses and have a cycle latency of 10 or more [6].…”

“…We postulate that automatic SoA transformations in a high-level language with constructs tailored to SIMD would allow developers to exploit the SIMD capabilities of CPUs in a convenient manner. With that in mind, we extend the SHAPES layout annotations [1] with programming concepts that are useful for writing programs with SIMD capabilities. This paper presents the main ideas and the design space.…”