High-speed, wide area, data intensive computing: a ten year retrospective

Abstract

Modern scientific computing involves organizing, moving, visualizing, and

Asymmetric communication models for resource‐constrained hierarchical ethernet networks

“…A tree topology platform: two star-configured racks connected via the backbone cable. The dashed arrows denote one example application with five logical communication links: e a 1 ;b 1 -e a 5 ;b 5 . The processes on each logical link are not explicitly labeled for clarity in the graph.…”

“…1576 J. ZHU ET AL.archy and resource sharing, make communication time prediction non-trivial and challenging for high-performance clusters.On the other hand, such predictions are needed more now than ever because of the increasing importance of data-intensive applications [4,5] that devote a significant amount of their total execution time in parallel processing to I/O or network communication, instead of computation. However, making such predictions accurately is nontrivial when contention exists on different components in hierarchical networks.…”

“…On the other hand, such predictions are needed more now than ever because of the increasing importance of data-intensive applications [4,5] that devote a significant amount of their total execution time in parallel processing to I/O or network communication, instead of computation. A good usable performance analysis of such data-intensive applications requires that the communication model reflects the network properties accurately on state-of-the-art network topologies and technologies.…”

“…. 1576 J. ZHU ET AL.archy and resource sharing, make communication time prediction non-trivial and challenging for high-performance clusters.On the other hand, such predictions are needed more now than ever because of the increasing importance of data-intensive applications [4,5] that devote a significant amount of their total execution time in parallel processing to I/O or network communication, instead of computation. A good usable performance analysis of such data-intensive applications requires that the communication model reflects the network properties accurately on state-of-the-art network topologies and technologies.In this article, we consider Ethernet-based network because, compared with custom interconnects (e.g., InfiniBand and Myrinet), it offers widespread compatibility, better cost-performance tradeoff, and a superior road map to 100-Gb standard [6,7].…”

Large-scale Real-time Data-driven Scientific Applications

“…In the spring of 1989, the concept, data intensive computing, was originated from a demonstration that would relate remote visualization and networking to prove the impact of high-speed networks by Craig Fields [3]. Purely data intensive applications process multi-terabyte to petabytes-sized datasets which commonly comes in several different formats and is often distributed across multiple locations [4].…”