Efficient Control and Communication Paradigms for Coarse-Grained Spatial Architectures

Abstract

There has been recent interest in exploring the acceleration of nonvectorizable workloads with spatially programmed architectures that are designed to efficiently exploit pipeline parallelism. Such an architecture faces two main problems: how to efficiently control each processing element (PE) in the system, and how to facilitate inter-PE communication without the overheads of traditional shared-memory coherent memory. In this article, we explore solving these problems using triggered instructions and latency-insensitive channels. Triggered instructions completely eliminate the program counter (PC) and allow programs to transition concisely between states without explicit branch instructions. Latency-insensitive channels allow efficient communication of inter-PE control information while simultaneously enabling flexible code placement and improving tolerance for variable events such as cache accesses. Together, these approaches provide a unified mechanism to avoid overserialized execution, essentially achieving the effect of techniques such as dynamic instruction reordering and multithreading. Our analysis shows that a spatial accelerator using triggered instructions and latency-insensitive channels can achieve 8 × greater area-normalized performance than a traditional general-purpose processor. Further analysis shows that triggered control reduces the number of static and dynamic instructions in the critical paths by 62% and 64%, respectively, over a PC-style baseline, increasing the performance of the spatial programming approach by 2.0 ×.

An Out-of-Order Load-Store Queue for Spatial Computing

“…Spatial or dataflow computing distributes the control and eliminates the requirement and expectation of static reasoning: each operation executes as soon as all of its inputs arrive, and physical operators pass along control "tokens" and the data they produce [16]. Different authors exploited latency-insensitive protocols to construct dynamic, high-performance circuits.…”

The Advantages and Challenges of Spatial Computation

“…Spatial computation is a paradigm that breaking the application's dataflow into regions, and these regions are mapped to some subset of the hardware resources, including functional units, interconnection network and storage, in the form of producer-consumer pipeline [7]. Spatial computation has some similarity with the multicore models.…”

“…In Fig. 2(a)(c), it shows the patterns of a program, totally parallelizable and loop-data-dependent, in which the circles with X y represents instruction X of iteration Y, and the squares with x y represents data read by instruction X Y [7]. Fig.…”

“…Triggered Instruction Architecture (TIA) [7] is a distributed controlled PE array, the on-chip connection is latency-intensive channel which is the foundation of the flexible architecture, which is similar with the architecture shown in Fig. 1.…”

dMazeRunner

“…For example, the convolution kernel (that executes for the majority of execution time in ResNeXt [1]) is a 7-deep perfectly nested loop. Variations of dataflow accelerators, like systolic arrays (e.g., Tensor Processing Unit), coarse-grained reconfigurable arrays (CGRAs), and spatial architectures are repeatedly being demonstrated as a promising accelerator for these power and performancecritical loops [2][3][4][5][6][7][8][9][10]. As shown in Figure 1, dataflow accelerators, in general, comprise an array of processing elements aka PEs (where PEs are function-units with little local control) and noncoherent scratchpad memories (SPM) that allow concurrent execution and explicit data management.…”