End-to-End Modeling and Optimization of Power Consumption in HPC Interconnects

Rumley, Sébastien; Polster, Robert; Hammond, Simon David; Rodrigues, Arun F.; Bergman, Keren

doi:10.1109/icppw.2016.33

Cited by 5 publications

(10 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At the time of writing this review, we are approximately at the point where, with current and emerging technology, optics is poised to provide at least modest energy reductions for data links compared to electrical approaches, even for relatively short links between cabinets and in backplanes or module connections [14], [15], [16], [17], [18], [19], [20].…”

Section: A Goals For This Reviewmentioning

confidence: 99%

“…Energy per bit References and notes Wireless data 10 -30μJ [31] Internet: access 40 -80nJ [8]; (a),(b) Internet: routing 20nJ [9]; (c) Internet: optical WDM links 3nJ [32]; (d) Reading DRAM 5pJ [5]; (e) Communicating off chip 1 -20 pJ [5], [15], [16] Data link multiplexing and timing circuits ~ 2 pJ [24] Communicating across chip 600 fJ [5]; (f) Floating point operation 100fJ [5]; (g) Energy in DRAM cell 10fJ [33]; (h) Switching CMOS gate ~50aJ -3fJ [4], [6], [34], [35]; (i) 1 electron at 1V, or 1 photon @1eV 0.16aJ (160zJ) WDM -wavelength division multiplexing DRAM -dynamic random-access memory CMOS -complementary metal-oxide-semiconductor transistor (a) Uses projections to 2016 from [8] (b) Presumes wired connections (optical or electrical) to the network (c) Total for 20 "hops" per internet connection, and derating energies from the 2008 numbers in [9] using a factor of 0.74 per year (from [8]) for improved electronics energy efficiency. (d) Total for 20 "hops" per internet connection, and using projections to 2016 in [32] (e) Rounded sum of the DRAM access and interface energies as projected for 2017 in [5], for off-chip DRAM (f) Based on 2017 projects in [5] for a 10mm line on the chip (g) Double-precision fused multiply-add (floating-point) operation using the projection in [5] of ~ 6.5 pJ in 2017 for this 64-bit operation to calculate energy per bit.…”

Section: Operationmentioning

confidence: 99%

“…As a result of these various factors, energies per bit for offchip electrical interconnects can typically range from picojoules per bit to significantly higher energies [5], [16]. This issue of off-chip connection energy and the difficulty in reducing it is well-known also in the context of supercomputers and their future scaling [15], for example.…”

Section: ) Energies For Electrical Off-chip Communicationsmentioning

confidence: 99%

See 2 more Smart Citations

Attojoule Optoelectronics for Low-Energy Information Processing and Communications

Miller

2017

J. Lightwave Technol.

605

406

View full text Add to dashboard Cite

Optics offers unique opportunities for reducing energy in information processing and communications while simultaneously resolving the problem of interconnect bandwidth density inside machines. Such energy dissipation overall is now at environmentally significant levels; the source of that dissipation is progressively shifting from logic operations to interconnect energies. Without the prospect of substantial reduction in energy per bit communicated, we cannot continue the exponential growth of our use of information. The physics of optics and optoelectronics fundamentally addresses both interconnect energy and bandwidth density, and optics may be the only scalable solution to such problems. Here we summarize the corresponding background, status, opportunities, and research directions for optoelectronic technology and novel optics, including sub-femtojoule devices in waveguide and novel 2D array optical systems. We compare different approaches to low-energy optoelectronic output devices and their scaling, including lasers, modulators and LEDs, optical confinement approaches (such as resonators) to enhance effects, and the benefits of different material choices, including 2D materials and other quantum-confined structures. With such optoelectronic energy reductions, and the elimination of line charging dissipation by the use optical connections, the next major interconnect dissipations are in the electronic circuits for receiver amplifiers, timing recovery and multiplexing. We show we can address these through the integration of photodetectors to reduce or eliminate receiver circuit energies, free-space optics to eliminate the need for timing and multiplexing circuits (while also solving bandwidth density problems), and using optics generally to save power by running large synchronous systems. One target concept is interconnects from ~ 1 cm to ~ 10 m that have the same energy (~ 10fJ/bit) and simplicity as local electrical wires on chip.

show abstract

Section: A Goals For This Reviewmentioning

confidence: 99%

Section: Operationmentioning

confidence: 99%

Section: ) Energies For Electrical Off-chip Communicationsmentioning

confidence: 99%

See 1 more Smart Citation

Attojoule Optoelectronics for Low-Energy Information Processing and Communications

Miller

2017

J. Lightwave Technol.

605

406

View full text Add to dashboard Cite

show abstract

“…For current systems [2], copper links at intra-rack distances consume approximately 3× less energy per bit compared to optical links (10-12 pJ/bit compared to 30-60 pJ/bit) [47]. We attempt rough exascale estimates based on previously reported extrapolations [4], [47]. For short-range electrical cables, we assume roughly 5pJ/bit at 1Tb/s for electrical and 10pJ/bit at 1Tb/s for optical cables (link + transceiver).…”

Section: B Cost Modelmentioning

confidence: 99%

“…To maintain feasibility and remain under a proposed 20MW power cap for the system, both the network procurement and pJ/bit of data movement costs must be controlled [3], [4]. The decreasing memory per core in high performance computing (HPC) also produces more off-chip traffic, further stressing the network.…”

Section: Introductionmentioning

confidence: 99%

APHiD: Hierarchical Task Placement to Enable a Tapered Fat Tree Topology for Lower Power and Cost in HPC Networks

Ibrahim

Shalf

Wilke

et al. 2017

2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)

View full text Add to dashboard Cite

Abstract-The power and procurement cost of bandwidth in system-wide networks has forced a steady drop in the byte/flop ratio. This trend of computation becoming faster relative to the network is expected to hold. In this paper, we explore how cost-oriented task placement enables reducing the cost of system-wide networks by enabling high performance even on tapered topologies where more bandwidth is provisioned at lower levels. We describe APHiD, an efficient hierarchical placement algorithm that uses new techniques to improve the quality of heuristic solutions and reduces the demand on high-level, expensive bandwidth in hierarchical topologies. We apply APHiD to a tapered fat-tree, demonstrating that APHiD maintains application scalability even for severely tapered network configurations. Using simulation, we show that for tapered networks APHiD improves performance by more than 50% over random placement and even 15% in some cases over costlier, state-of-the-art placement algorithms.

show abstract