Precise automatable analytical modeling of the cache behavior of codes with indirections

Abstract

The performance of memory hierarchies, in which caches play an essential role, is critical in nowadays general-purpose and embedded computing systems because of the growing memory bottleneck problem. Unfortunately, cache behavior is very unstable and difficult to predict. This is particularly true in the presence of irregular access patterns, which exhibit little locality. Such patterns are very common, for example, in applications in which pointers or compressed sparse matrices give place to indirections. Nevertheless, cache behavior in the presence of irregular access patterns has not been widely studied. In this paper we present an extension of a systematic analytical modeling technique based on PMEs (probabilistic miss equations), previously developed by the authors, that allows the automated analysis of the cache behavior for codes with irregular access patterns resulting from indirections. The model generates very accurate predictions despite the irregularities and has very low computing requirements, being the first model that gathers these desirable characteristics that can automatically analyze this kind of codes. These properties enable this model to help drive compiler optimizations, as we show with an example.

Cache Behavior Modelling for Codes Involving Banded Matrices

“…The situation depicted in our example is clearly more common than the one modeled in our previous work [8], in which we only considered irregular access patterns which had an uniform probability of access for each element of the dereferenced data structure, and in which such probability did not change during the execution of the code. It is very usual that the diagonals of banded matrices have different densities, with the distribution of the nonzeros within each diagonal being relatively uniform.…”

“…The shape of PME F Ri depends on the access pattern followed by R in loop i. The PME formulas for references following a regular access pattern and references following irregular access patterns due to indirections with a uniform distribution have been presented in [8]. A simple extension was also proposed in [8] to support accesses generated by the processing of banded matrices with an uniform distribution of the entries inside the band by applying small modifications to the formulas of indirections with uniform distributions.…”

“…The calculation of the memory regions accessed during a reuse distance is not covered in this paper due to space limitations (see [8] for more information). Still, it is worth to comment that the PME model maps the regions into a mathematical representation consisting of a vector V of k+1 probabilities, where k is the associativity of the cache, called area vector.…”

“…3 Scope of Application of the PME Model Figure 1 shows the kind of codes covered by the PME model extended to cope with irregular access patterns due to indirections [8]. It is a set of perfectly or non-perfectly nested loops in which the number of iterations of the loops must be known at compile-time and the indexing of the data structures must be done using either affine functions of the loop indexes or across indirections.…”

“…It is a set of perfectly or non-perfectly nested loops in which the number of iterations of the loops must be known at compile-time and the indexing of the data structures must be done using either affine functions of the loop indexes or across indirections. In [8] the values generated by the indirection had to follow an uniform distribution, that is, every position along the indexed dimension had to have the same probability of being accessed. Unfortunately this situation is not very common.…”