The data locality of work stealing

High Performance Datacenter Networks: Architectures, Algorithms, and Opportunities

“…2 The actual hardware also has separate physical memories as shown in Figure 1.1: CPUs have their own DRAM and GPUs also have their own DRAM. The GPU's DRAM memory space is called global memory or device memory.…”

“…Since there are n 2 elements of C, the work W (n) = (n 3 ). 2 To compute D(n), observe that the n 2 output elements have no dependencies among them; the only dependencies occur during the reduction to compute each output element. Thus, D(n) = (log n).…”

“…with high probability [2]. One might have reasonably guessed an upper-bound of p · Q 1 (n; Z, L), i.e., that the algorithm could incur as many as p times more misses in the multithreaded case, but Equation (2.2) shows otherwise.…”

“…The algorithm is complex and its details beyond the scope of this synthesis lecture; we instead refer the interested reader elsewhere for details [43,101]. 2 For simplicity, we assume the number of source and target points is the same, though in general they may differ. Moreover, the sets of points may in fact be exactly the same, with the sum excluding the computed target.…”

“…2 The roofline sets an upper bound on the performance of a kernel, depending on the kernel's operational intensity. When operational intensity as a column hits the roof, either it hits the flat part of the roof, which means performance is compute bound, or performance is ultimately memory bound.…”