Automatically enhancing locality for tree traversals with traversal splicing

Abstract

Generally applicable techniques for improving temporal locality in irregular programs, which operate over pointerbased data structures such as trees and graphs, are scarce. Focusing on a subset of irregular programs, namely, tree traversal algorithms like Barnes-Hut and nearest neighbor, previous work has proposed point blocking, a technique analogous to loop tiling in regular programs, to improve locality. However point blocking is highly dependent on point sorting, a technique to reorder points so that consecutive points will have similar traversals. Performing this a priori sort requires an understanding of the semantics of the algorithm and hence highly application specific techniques. In this work, we propose traversal splicing, a new, general, automatic locality optimization for irregular tree traversal codes, that is less sensitive to point order, and hence can deliver substantially better performance, even in the absence of semantic information. For six benchmark algorithms, we show that traversal splicing can deliver singlethread speedups of up to 9.147 (geometric mean: 3.095) over baseline implementations, and up to 4.752 (geometric mean: 2.079) over point-blocked implementations. Further, we show that in many cases, automatically applying traversal splicing to a baseline implementation yields performance that is better than carefully hand-optimized implementations.

Locality Transformations for Nested Recursive Iteration Spaces

“…In recent years, there has been more work targeting locality transformations for recursive programs, including dynamic pointer alignment [32], point blocking [20], traversal splicing [21]. These transformations are, effectively, variants of loop tiling in various ways for recursion that is nested within a normal, iterative iteration space (specifically, for loops).…”

“…In the past five years, there have been some attempts to expand the space of locality-enhancing transformations to work on recursive iteration spaces 1 . For example, Jo and Kulkarni look at transformations on recursive iteration spaces nested within an iterative iteration space, in the context of repeated traversals of trees [20,21]; the outer loop iterates over points, and the inner recursive traversal of the tree generates a recursive iteration space. Rajbhandari et al look at fusing two (or more) recursive iteration spaces, in an analog of loop fusion [25,26]; here the separate recursive iteration spaces represent separate traversals of a kd-tree, and the fusion transformation yields a single recursive traversal of the tree.…”

Hybrid CPU-GPU scheduling and execution of tree traversals

“…For Barnes-Hut alone, researchers have proposed sorting using space-filling curves [2], Zcurves [9], orthogonal bisection [26], or the structure of the Barnes-Hut tree itself [28]. For ray tracers, researchers have suggested various ray-reorganization techniques [21,20,19,1,18] Rather than devising new sorting strategies for each new traversal algorithm, several researchers have looked at using the past behavior of computations to predict their future tree accesses, and hence dynamically schedule them with minimal application-specific knowledge [31,22,13,12]. Most directly relevant, as they target the same types of algorithms as this paper, is Jo and Kulkarni's traversal splicing work [13,12].…”

“…The insight that past traversal behavior is correlated with future behavior has been exploited before, in narrower, or applicationspecific contexts by Zhang et al [31] and Pingali et al [22], and in a more general tree-traversal context by Jo and Kulkarni [13,12]. Fundamentally, these approaches all interleave the scheduling component (tracking past behavior and reorganizing computations based on that behavior) with the execution (perform the next phase of computations).…”

Locality Transformations for Nested Recursive Iteration Spaces

“…In the past five years, there have been some attempts to expand the space of locality-enhancing transformations to work on recursive iteration spaces 1 . For example, Jo and Kulkarni look at transformations on recursive iteration spaces nested within an iterative iteration space, in the context of repeated traversals of trees [20,21]; the outer loop iterates over points, and the inner recursive traversal of the tree generates a recursive iteration space. Rajbhandari et al look at fusing two (or more) recursive iteration spaces, in an analog of loop fusion [25,26]; here the separate recursive iteration spaces represent separate traversals of a kd-tree, and the fusion transformation yields a single recursive traversal of the tree.…”