A Theoretical Framework for Algorithm-Architecture Co-design

Czechowski, Kenneth; Vuduc, Richard

doi:10.1109/ipdps.2013.99

Cited by 23 publications

(21 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We instantiate and validate this model experimentally on existing systems. An intuitive interpretation of the model in terms of processor balance [6,8,11,33,35] is given in § 4. …”

Section: Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

On the communication complexity of 3D FFTs and its implications for Exascale

Czechowski

Battaglino

McClanahan

et al. 2012

Proceedings of the 26th ACM International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

This paper revisits the communication complexity of largescale 3D fast Fourier transforms (FFTs) and asks what impact trends in current architectures will have on FFT performance at exascale. We analyze both memory hierarchy traffic and network communication to derive suitable analytical models, which we calibrate against current software implementations; we then evaluate models to make predictions about potential scaling outcomes at exascale, based on extrapolating current technology trends. Of particular interest is the performance impact of choosing high-density processors, typified today by graphics co-processors (GPUs), as the base processor for an exascale system. Among various observations, a key prediction is that although inter-node all-to-all communication is expected to be the bottleneck of distributed FFTs, intra-node communication-expressed precisely in terms of the relative balance among compute capacity, memory bandwidth, and network bandwidth-will play a critical role.

show abstract

“…We instantiate and validate this model experimentally on existing systems. An intuitive interpretation of the model in terms of processor balance [6,8,11,33,35] is given in § 4. …”

Section: Contributionsmentioning

confidence: 99%

“…This factor is the classical definition of balance [6,8,11,33,35], which has units of flops / word (or byte), applied here to node performance. When this factor is large (in this case relative to the inherent flop:byte requirements of an FFT), we say the node is imbalanced.…”

Section: Interpreting the Model: Balanced Processorsmentioning

confidence: 99%

On the communication complexity of 3D FFTs and its implications for Exascale

Czechowski

Battaglino

McClanahan

et al. 2012

Proceedings of the 26th ACM International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

show abstract

“…To further reduce the number of runs needed to derive the model parameters, future work might combine the modeling approach proposed in this paper with the general modeling approach developed in our prior work [44,45], thereby yielding power models that are both accurate and generic.…”

Section: Discussionmentioning

confidence: 99%

Modeling the Power Variability of Core Speed Scaling on Homogeneous Multicore Systems

Lee

et al. 2017

Scientific Programming

Self Cite

View full text Add to dashboard Cite

We describe a family of power models that can capture the nonuniform power effects of speed scaling among homogeneous cores on multicore processors. These models depart from traditional ones, which assume that individual cores contribute to power consumption as independent entities. In our approach, we remove this independence assumption and employ statistical variables of core speed (average speed and the dispersion of the core speeds) to capture the comprehensive heterogeneous impact of subtle interactions among the underlying hardware. We systematically explore the model family, deriving basic and refined models that give progressively better fits, and analyze them in detail. The proposed methodology provides an easy way to build power models to reflect the realistic workings of current multicore processors more accurately. Moreover, unlike the existing lowerlevel power models that require knowledge of microarchitectural details of the CPU cores and the last level cache to capture core interdependency, ours are easier to use and scalable to emerging and future multicore architectures with more cores. These attributes make the models particularly useful to system users or algorithm designers who need a quick way to estimate power consumption. We evaluate the family of models on contemporary x86 multicore processors using the SPEC2006 benchmarks. Our best model yields an average predicted error as low as 5%.

show abstract

“…In practice, the prohibitive problem size prevents direct solution of (11). Hence, various iterative methods were developed to efficiently solve such large linear systems.…”

Section: Inriamentioning

confidence: 99%

“…Czechowski et al [11,12] consider the relationship between the ratio of an algorithm's data movement cost to arithmetic work and the machine balance ratio of memory bandwidth to peak performance. Our analysis of algorithms in this paper involves a very similar theme as theirs and is inspired by their work, but we develop new lower bounds analysis to perform the analysis.…”

Section: Related Workmentioning

confidence: 99%

On characterizing the data movement complexity of computational DAGs for parallel execution

Elango

Rastello

Pouchet

et al. 2014

Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

Abstract:Technology trends are making the cost of data movement increasingly dominant, both in terms of energy and time, over the cost of performing arithmetic operations in computer systems. The fundamental ratio of aggregate data movement bandwidth to the total computational power (also referred to the machine balance parameter ) in parallel computer systems is decreasing. It is therefore of considerable importance to characterize the inherent data movement requirements of parallel algorithms, so that the minimal architectural balance parameters required to support it on future systems can be well understood. In this paper, we develop an extension of the well-known red-blue pebble game to develop lower bounds on the data movement complexity for the parallel execution of computational directed acyclic graphs (CDAGs) on parallel systems. We model multi-node multi-core parallel systems, with the total physical memory distributed across the nodes (that are connected through some interconnection network) and in a multi-level shared cache hierarchy for processors within a node. We also develop new techniques for lower bound characterization of non-homogeneous CDAGs. We demonstrate the use of the methodology by analyzing the CDAGs of several numerical algorithms, to develop lower bounds on data movement for their parallel execution.

show abstract

A Theoretical Framework for Algorithm-Architecture Co-design

Cited by 23 publications

References 47 publications

On the communication complexity of 3D FFTs and its implications for Exascale

On the communication complexity of 3D FFTs and its implications for Exascale

Modeling the Power Variability of Core Speed Scaling on Homogeneous Multicore Systems

On characterizing the data movement complexity of computational DAGs for parallel execution

Contact Info

Product

Resources

About