Data reshuffling in support of fast I/O for distributed-memory machines

Bornstein, Claudson F.; Steenkiste, Peter

doi:10.1109/hpdc.1994.340240

Cited by 5 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We observe that both algorithms easily support throughputs well in excess of the target HIPPI rates. The reason is that the reshuffling algorithms are highly parallel: they exploit both parallelism between rows in the iWarp system and inside each row of iWarp [Bornstein and Steenkiste 1994]. Figure 15 shows the throughput with which we can transfer data to the HIB, starting with data mapped on the system using distributions with different block sizes.…”

Section: Performance Measurementsmentioning

confidence: 98%

“…This architecture does not imply that each application has to provide the code to transfer data to and from the network interface. For example, libraries can be built for common data distributions (e.g., Bornstein and Steenkiste [1994]). …”

Section: Architecturementioning

confidence: 99%

“…The I/O mapping chosen corresponds to a block-cyclic distribution. The algorithm used to reshuffle data from an arbitrary block-cyclic distribution to the I/O distribution is described in Bornstein and Steenkiste [1994]. …”

Section: Common Data Distributionsmentioning

confidence: 99%

See 2 more Smart Citations

A high-speed network interface for distributed-memory systems

Steenkiste

1997

ACM Trans. Comput. Syst.

View full text Add to dashboard Cite

Distributed-memory systems have traditionally had great difficulty performing network I/O at rates proportional to their computational power. The problem is that the network interface has to support network I/O for a supercomputer, using computational and memory bandwidth resources similar to those of a workstation. As a result, the network interface becomes a bottleneck. In this article we present an I/O architecture that addresses these problems and supports high-speed network I/O on distributed-memory systems. The key to good performance is to partition the work appropriately between the system and the network interface. Some communication tasks are performed on the distributed-memory parallel system, since it is more powerful and less likely to become a bottleneck than the network interface. Tasks that do not parallelize well are performed on the network interface, and hardware support is provided for the most time-critical operations. This architecture has been implemented for the iWarp distributed-memory system and has been used by a number of applications. We describe this implementation, present performance results, and use application examples to validate the main features of the I/O architecture.

show abstract

Section: Performance Measurementsmentioning

confidence: 98%

Section: Architecturementioning

confidence: 99%

See 1 more Smart Citation

A high-speed network interface for distributed-memory systems

Steenkiste

1997

ACM Trans. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…In the approach we selected [8], the distributed-memory system is responsible for the scatter/gather operation. On transmit, it constructs large messages and presents them to the interface in an efficient way, for example striped across multiple links.…”

Section: Data Distributionmentioning

confidence: 99%

Gigabit Nectar: Architecture and Performance.

Steenkiste¹

1995

View full text Add to dashboard Cite

show abstract

Introduction to Multiprocessor I/O Architecture

Kotz

1996

The Kluwer International Series in Engineering and Computer Science

View full text Add to dashboard Cite

The computational performance of multiprocessors continues to improve b y leaps and bounds, fueled in part by rapid improvements in processor and interconnection technology. I/O performance thus becomes ever more critical, to avoid becoming the bottleneck of system performance. In this paper we p r o vide an introduction to I/O architectural issues in multiprocessors, with a focus on disk subsystems. While we discuss examples from actual architectures and provide pointers to interesting research in the literature, we do not attempt to provide a comprehensive s u r v ey. W e concentrate on a study of the architectural design issues, and the e ects of di erent design alternatives.

show abstract

Data reshuffling in support of fast I/O for distributed-memory machines

Cited by 5 publications

References 16 publications

A high-speed network interface for distributed-memory systems

A high-speed network interface for distributed-memory systems

Gigabit Nectar: Architecture and Performance.

Introduction to Multiprocessor I/O Architecture

Contact Info

Product

Resources

About