Improving data locality with loop transformations

Abstract

In the past decade, processor speed has become significantly faster than memory speed. Small, fast cache memories are designed to overcome this discrepancy, but they are only effective when programs exhibit data locality. In this article, we present compiler optimizations to improve data locality based on a simple yet accurate cost model. The model computes both temporal and spatial reuse of cache lines to find desirable loop organizations. The cost model drives the application of compound transformations consisting of loop permutation, loop fusion, loop distribution, and loop reversal. We demonstrate that these program transformations are useful for optimizing many programs. To validate our optimization strategy, we implemented our algorithms and ran experiments on a large collection of scientific programs and kernels. Experiments illustrate that for kernels our model and algorithm can select and achieve the best loop structure for a nest. For over 30 complete applications, we executed the original and transformed versions and simulated cache hit rates. We collected statistics about the inherent characteristics of these programs and our ability to improve their data locality. To our knowledge, these studies are the first of such breadth and depth. We found performance improvements were difficult to achieve because benchmark programs typically have high hit rates even for small data caches; however, our optimizations significantly improved several programs.

A Case Study of a Navigator Optimization Process

“…In a real-time embedded system development, a key part in determining system performance is related to memory access latency [6] [7]. In design step, this problem can be changed by hardware, but it is not easy to improve the memory system performance at the time when the hardware part development is completed.…”

Loop Distribution and Fusion with Timing and Code Size Optimization for Embedded DSPs

“…Since loops are prevalent in multimedia processing, digital signal processing, etc., loop transformations such as loop fusion, loop distribution are needed to optimize the execution of the loops in embedded DSP applications [8,[1][2][3]5].…”

“…Loop distribution separates independent statements inside a single loop (or loop nest) into multiple loops (or loop nests) [8,1,3,2,5]. Loop distribution can be used to break up a large loop that doesn't fit into the cache [2,3,8,1].…”

“…Loop distribution can also enable other loop optimization techniques such as loop fusion, loop interchanging and loop permutation [8,1,3,5].…”

“…There are a lot of previous works using loop fusion to optimize the execution of loops [2,3,8,1,4,5,7]. But sometimes loop shifting or retiming is needed to enable loop fusion, which will cause the code size expansion because of the generation of the prologue and epilogue.…”